moDiNA
Submodules
modina.context_net_inference module
- modina.context_net_inference.calculate_association_scores(ord_data, nom_data, cont_data, bi_data, test_type='nonparametric', num_workers=1, nan_value=-89.0, correction='bh', n_min_valid=10, correction_scope='global')[source]
- Return type:
DataFrame
- modina.context_net_inference.compute_context_scores(context_data, meta_file, test_type='nonparametric', correction='bh', num_workers=1, path=None, nan_value=None, name='context1', n_min_valid=10, correction_scope='global')[source]
Compute association scores for a given context.
- Parameters:
context_data (
DataFrame) – Raw context data (rows: samples, columns: variables).meta_file (
DataFrame) – Metadata file containing a ‘label’ and ‘type’ column to specify the data type of each variable.test_type (
str) – Type of tests to use for network inference. Defaults to ‘nonparametric’.correction (
str) – Correction method for multiple testing. Defaults to ‘bh’.num_workers (
int) – Number of workers for parallel processing. Defaults to 1.path (
Optional[str]) – Optional path to save the computed scores as a CSV file. Defaults to None.nan_value (
Optional[float]) – Numerical value used for NaN values in the context data. If None, an error will be raised if such values are present. Defaults to None.name (
str) – Name of the context. Used for saving files. Defaults to ‘context’.n_min_valid (
int) – Minimum number of samples (after pairwise removal of missing values) a pairwise test must be based on. Pairs below this are excluded from this context’s network (and, during differential analysis, from the other context’s as well). Defaults to 10.correction_scope (
str) – Scope of the multiple-testing correction. ‘global’ pools all edges into one correction family; ‘local’ corrects separately within each statistical test type (e.g. Pearson, Spearman, t-test, Mann-Whitney U, chi-squared, Kruskal-Wallis). Defaults to ‘global’.
- Return type:
DataFrame- Returns:
A pd.DataFrame containing the computed association scores.
- modina.context_net_inference.napy_bi_cont(cont_phenotypes, bi_phenotypes, test='nonparametric', num_workers=8, nan_value=-89.0)[source]
- modina.context_net_inference.napy_bi_nom(nom_phenotypes, bi_phenotypes, num_workers=8, nan_value=-89.0)[source]
- modina.context_net_inference.napy_bi_ord(ord_phenotypes, bi_phenotypes, num_workers=8, nan_value=-89.0)[source]
- modina.context_net_inference.napy_cont_cont(cont_phenotypes, test='nonparametric', num_workers=8, nan_value=-89.0)[source]
- modina.context_net_inference.napy_nom_cont(cont_phenotypes, nom_phenotypes, test='nonparametric', num_workers=8, nan_value=-89.0)[source]
modina.context_simulation module
- modina.context_simulation.save_gt(groundtruths, path, mode='node', edge_magnitude=None, node_stats=None, node_shift_magnitude=None)[source]
Write ground truth nodes or edges to a two- (or, with the optional data below, wider-) column file.
- Parameters:
groundtruths – (shift_nodes, corr_nodes, shift_corr_nodes) as returned by simulate_copula.
path – File to write.
mode – ‘node’ writes one row per (node, description); ‘edge’ writes one row per (edge, description), edge = ‘_’.join(sorted(pair)).
edge_magnitude – Optional, mode=’edge’ only. Dict keyed by frozenset({node1, node2}) -> the correlation magnitude injected on that edge. When given, an extra corr column is appended. Omit to reproduce the historical two-column file exactly.
node_stats – Optional, mode=’node’ only. Dict node -> (n_edges_tweaked, sum_corr). When given, two extra columns are appended; a node absent from the dict (e.g. mean-shift-only) is written as (0, 0.0). Omit to reproduce the historical two-column file exactly.
node_shift_magnitude – Optional, mode=’node’ only. Dict node -> mean-shift magnitude injected on that node (e.g. effects[‘node_shift_magnitude’]). When given, an extra shift column is appended; a node with no mean-shift draw of its own (e.g. correlation-only) is written as 0.0. Omit to reproduce the file exactly as it was before this column existed.
The magnitude columns are named shift and corr after the quantity they hold, matching the shift_low/shift_high and corr_low/corr_high parameters the values are drawn from.
- modina.context_simulation.simulate_copula(path=None, name1='context1', name2='context2', n_bi=50, n_cont=50, n_cat=50, n_samples_1=500, n_samples_2=500, n_shift_cont=0, n_shift_bi=0, n_shift_cat=0, n_corr_cont_cont=0, n_corr_bi_bi=0, n_corr_cat_cat=0, n_corr_bi_cont=0, n_corr_bi_cat=0, n_corr_cont_cat=0, n_both_cont_cont=0, n_both_bi_bi=0, n_both_cat_cat=0, n_both_bi_cont=0, n_both_bi_cat=0, n_both_cont_cat=0, *, corr_low, corr_high, shift_low, shift_high, hub_reuse_prob=0.0, background_strength=0.2, background_density=2, background_seed=None, background_model='modules', module_size=None, module_fraction=0.1, module_correlation=0.2)[source]
Simulate two contexts with binary and continuous nodes using a Gaussian copula.
- Parameters:
path – Path to save the simulated contexts, the meta file and the ground truth information. If None, files are not saved.
name1 – Name of the first context.
name2 – Name of the second context.
n_bi – Number of binary nodes to simulate.
n_cont – Number of continuous nodes to simulate.
n_cat – Number of categorical nodes to simulate.
n_samples_1 – Number of samples for the first context.
n_samples_2 – Number of samples for the second context.
n_shift_cont – Number of continuous nodes with an artificially introduced mean shift.
n_shift_bi – Number of binary nodes with an artificially introduced mean shift.
n_shift_cat – Number of categorical nodes with an artificially introduced mean shift.
n_corr_cont_cont – Number of continuous node pairs with an artifically introduced correlation difference.
n_corr_bi_bi – Number of binary node pairs with an artificially introduced correlation difference.
n_corr_cat_cat – Number of categorical node pairs with an artificially introduced correlation difference.
n_corr_bi_cat – Number of binary-categorical node pairs with an artificially introduced correlation difference.
n_corr_cont_cat – Number of continuous-categorical node pairs with an artificially introduced correlation difference.
n_corr_bi_cont – Number of mixed node pairs with an artificially introduced correlation difference.
n_both_cont_cont – Number of continuous node pairs with both an aritificially introduced mean shift and correlation difference.
n_both_bi_bi – Number of binary node pairs with both an artificially introduced mean shift and correlation difference.
n_both_cat_cat – Number of categorical node pairs with both an artificially introduced mean shift and correlation difference.
n_both_bi_cat – Number of binary-categorical node pairs with both an artificially introduced mean shift and correlation difference.
n_both_cont_cat – Number of continuous-categorical node pairs with both an artificially introduced mean shift and correlation difference.
n_both_bi_cont – Number of mixed node pairs with both an artificially introduced mean shift and correlation difference.
corr_low – Lower bound of the correlation-difference magnitude range. Each correlated pair independently draws its actual magnitude uniformly from [corr_low, corr_high], so ground truth effects span a continuum – rankable by realized magnitude – rather than clustering at one fixed value. Required.
corr_high – Upper bound of the correlation-difference magnitude range. Must satisfy corr_low <= corr_high <= 0.95 (0.95 is the historical ceiling on a single edge’s magnitude). Required.
shift_low – Lower bound of the mean-shift magnitude range. Each shifted node independently draws its actual magnitude uniformly from [shift_low, shift_high], for the same reason as corr_low/corr_high. Required.
shift_high – Upper bound of the mean-shift magnitude range. Must satisfy shift_low <= shift_high. Required.
hub_reuse_prob – Fraction of each pair type’s correlation/both pairs that reuse an already-placed node as one endpoint (“hub”) instead of drawing two fresh nodes. Real differential networks concentrate rewiring on a few hub nodes rather than a perfect matching. The count per pair type is fixed at round(hub_reuse_prob * n_pairs) rather than drawn per edge, and cross-type reuses are spread so every node type loses the same share of its endpoints to hubs (see _plan_hub_reuse); this keeps the number of ground truth nodes per type consistent across replicates and types. A reused hub is picked at random among the eligible nodes of its type whose running sum(magnitude**2) stays below the background’s lambda_min**2. A reuse only ever fills one slot of a pair; the other is always a fresh node, and that fresh node – plus the hub’s original first partner, the first time the hub is reused – is permanently barred from ever being reused itself. This keeps every hub isolated from every other hub, which is what makes the sum(magnitude**2) check exact. If no eligible hub exists yet, the reuse is deferred to the next fresh pair of the same pair type. Default 0.0 gives one edge per node.
The background correlation structure is built ONCE and shared by both contexts, so every pair that is not planted has bit-identical correlation in the two contexts and the negatives stay exactly null. By default (background_model=’modules’) it consists of correlated modules (see module_fraction). The alternative, background_model=’scale_free’, is a sparse scale-free network (preferential attachment, producing a few high-degree hubs and a heavy-tailed degree distribution), turned into a dense correlation matrix (see background_strength and background_density). :type background_strength: :param background_strength: Only used with background_model=’scale_free’. Target mean |r| over
the directly-wired pairs of the background. This is a property of the resulting correlation matrix, not a value written anywhere: the diagonal loading that attains it is solved for by bisection. It is bounded above by the topology (denser wiring forces weaker correlations); an unreachable request raises with the attainable ceiling.
- Parameters:
background_density – Only used with background_model=’scale_free’. Number of connections each node makes when it is added to the network. Sets both the edge count and how large the hubs grow. Higher density lowers the reachable strength.
background_seed – Seed for the background’s own random generator. Kept separate from the global streams the rest of the simulation draws from, so changing the background does not change which pairs get planted or with what magnitudes.
background_model – ‘modules’ (default) builds a dense background of correlated modules (see _build_module_background) from module_fraction (or module_size) and module_correlation, and ignores the two scale-free parameters. ‘scale_free’ builds the scale-free background from background_strength and background_density. The module background reaches the association density of real clinical cohorts, which the scale-free one cannot: its diagonally dominant precision matrix caps the share of pairs with |r| > 0.15 at about 1%.
module_size – Number of variables per module (modules have near-equal sizes). Default None: set from module_fraction. Only used with background_model=’modules’.
module_fraction – Module size as a share of all variables, in (0, 1], used when module_size is None: module_size = max(2, round(module_fraction * n_vars)). Scaling the modules with the number of variables keeps the background density constant across variable counts. Default 0.1. Only used with background_model=’modules’.
module_correlation – Correlation between the module factors, in [0, 1). Sets the weak associations between modules and with them how dense the background is. Default 0.2. Only used with background_model=’modules’.
- Returns:
A tuple containing the two simulated contexts, a meta file, a list of ground truth nodes and a dict of per-edge/per-node injected effect sizes. - context1: pd.DataFrame of the first simulated context. - context2: pd.DataFrame of the second simulated context. - meta: pd.DataFrame containing the data type for each simulated variable. - ground_truth: A tuple containing three lists of ground truth nodes: (shift_nodes, corr_nodes, shift_corr_nodes). - effects: dict with ‘edge_magnitude’ (dict keyed by frozenset({node1, node2}) -> the
jittered correlation magnitude actually injected on that edge), ‘node_degree’ (dict node -> number of correlation edges touching it) and ‘node_sum_magnitude’ (dict node -> sum of jittered magnitudes over those edges). Feeds the extra columns written to ground_truth_edges.txt/ground_truth_nodes.txt when path is given, and lets a caller that writes those files itself (as the Nextflow CLI wrapper does) do the same. Also carries ‘node_shift_magnitude’ (dict node -> its drawn mean-shift magnitude, 0.0 for nodes with no mean-shift draw of their own), ‘corr_base’ (the shared background correlation matrix), ‘background_edges’ (the directly-wired pairs, as node-name frozensets) and ‘background_lambda_min’ (the perturbation budget the planting step was checked against).
modina.diff_net_construction module
- modina.diff_net_construction.compute_diff_edges(scores1, scores2, edge_metric, path=None, name1='context1', name2='context2')[source]
Compute differential edge scores based on the specified edge metric.
Every edge metric is built from per-context scores (see statistics_utils.add_score_columns): ‘diff-<score>’ is the absolute between-context difference of one score (P, LP, RE, VE, LPxRE or LPxVE), and ‘sum-LP-VE’ / ‘sum-LP-RE’ add up the absolute differences of LP and VE / RE. ‘diff-Z’ is the change of the standardised effect size z / sqrt(n) in units of its standard error (see _diff_z); it needs the per-edge sample size column ‘n’ in both score tables.
- Parameters:
scores1 (
DataFrame) – Statistical association scores of Context 1, rescaled and potentially filtered.scores2 (
DataFrame) – Statistical association scores of Context 2, rescaled and potentially filtered.edge_metric (
str) – Edge metric to compute the differential edge scores.path (
Optional[str]) – Optional path to save the differential edge scores as a CSV file. Defaults to None.name1 (
str) – Name of Context 1, used to label the per-contextraw-P/raw-Ecolumns. Defaults to ‘context1’.name2 (
str) – Name of Context 2, used to label the per-contextraw-P/raw-Ecolumns. Defaults to ‘context2’.
- Return type:
Tuple[DataFrame,DataFrame]- Returns:
A tuple (edges_diff, edge_node_stats): the computed differential edge scores and a node-indexed DataFrame of per-node statistics over incident edges (see edge_node_statistics).
- modina.diff_net_construction.compute_diff_network(scores1, scores2, context1, context2, edge_metric=None, node_metric=None, correction='bh', correction_scope='global', num_workers=1, path=None, format='csv', meta_file=None, test_type='nonparametric', nan_value=None, name1='context1', name2='context2')[source]
Computation of a differential network defined by a node metric and an edge metric.
- Parameters:
scores1 (
DataFrame) – Statistical association scores of Context 1, rescaled and potentially filtered.scores2 (
DataFrame) – Statistical association scores of Context 2, rescaled and potentially filtered.context1 (
DataFrame) – Observed data of Context 1, potentially filtered.context2 (
DataFrame) – Observed data of Context 2, potentially filtered.edge_metric (
Optional[str]) – Edge metric used to construct the differential network.node_metric (
Optional[str]) – Node metric used to construct the differential network.correction (
str) – Correction method for multiple testing. Defaults to ‘bh’.correction_scope (
str) – Scope of the multiple-testing correction for STC: ‘global’ (all nodes in one family) or ‘local’ (nodes corrected separately per test type used to compare them across contexts). Defaults to ‘global’.num_workers (
int) – Number of workers for parallel computation of STC. Defaults to 1.path (
Optional[str]) – Optional path to save the differential scores as CSV files. Defaults to None.format (
str) – File format to save the differential network. Options are ‘csv’ and ‘graphml’. Defaults to ‘csv’.meta_file (
Optional[DataFrame]) – Meta file containing the node types. Only needed for the STC node metrics. Defaults to None.test_type (
str) – Test type to use for continuous nodes in the STC node metrics. Defaults to ‘nonparametric’.nan_value (
Optional[int]) – Numerical value used for NaN values in the context data. If None, an error will be raised if such values are present. Defaults to None.
- Return type:
Tuple[DataFrame|None,DataFrame|None,DataFrame|None]- Returns:
A tuple (edges_diff, nodes_diff, edge_node_stats) containing the computed differential edges, differential nodes, and per-node statistics over incident edges (None if no edge_metric).
- modina.diff_net_construction.compute_diff_nodes(scores1, scores2, context1, context2, node_metric, correction='bh', meta_file=None, test_type='nonparametric', nan_value=None, num_workers=1, path=None, correction_scope='global')[source]
Compute differential node scores based on the specified node metric.
Node metrics are ‘<family>-<score>’ with family WDC (weighted degree centrality), PRC (PageRank centrality) or STC (statistical test centrality) and score P, LP, RE, VE, LPxRE or LPxVE (see statistics_utils.add_score_columns). WDC and PRC use the score as edge weight in each context and take the absolute difference of the max-normalised centralities; STC tests each node between the contexts and derives the score from the test’s adjusted p-value and effect size.
- Parameters:
scores1 (
DataFrame) – Statistical association scores of Context 1, rescaled and potentially filtered.scores2 (
DataFrame) – Statistical association scores of Context 2, rescaled and potentially filtered.context1 (
DataFrame) – Observed data of Context 1, potentially filtered.context2 (
DataFrame) – Observed data of Context 2, potentially filtered.node_metric (
str) – Node metric to compute the differential node scores.correction (
str) – Correction method for multiple testing. Only needed for the STC node metrics. Defaults to ‘bh’.meta_file (
Optional[DataFrame]) – Meta file containing the node types. Only needed for the STC node metrics. Defaults to None.test_type (
str) – Test type to compare continuous variables across contexts for the STC node metrics. Defaults to ‘nonparametric’.nan_value (
Optional[int]) – Numerical value used for NaN values in the context data. If None, an error will be raised if such values are present. Defaults to None.num_workers (
int) – Number of workers for parallel computation of STC. Only needed for the STC node metrics. Defaults to 1.path (
Optional[str]) – Optional path to save the differential node scores as a CSV file. Defaults to None.correction_scope (
str) – Scope of the multiple-testing correction for STC: ‘global’ (all nodes in one family) or ‘local’ (nodes corrected separately per test type used to compare them across contexts). Defaults to ‘global’.
- Return type:
DataFrame- Returns:
A DataFrame containing the computed differential node scores.
- modina.diff_net_construction.degree_centrality(nodes_diff, scores1, scores2, metric='WDC-P')[source]
- modina.diff_net_construction.edge_node_statistics(edges_diff, edge_metric)[source]
Compute summary statistics of the (absolute) edge metric over the edges incident to each node.
- Parameters:
edges_diff (
DataFrame) – Differential edge scores with columns ‘label1’, ‘label2’ and edge_metric.edge_metric (
str) – Name of the (absolute) edge metric column.
- Return type:
DataFrame- Returns:
A node-indexed DataFrame with columns ‘edge-min’, ‘edge-max’, ‘edge-median’, ‘edge-mean’, ‘edge-sd’ and ‘edge-percentile-mean’. sd is the sample standard deviation (ddof=1) and is NaN for nodes with a single incident edge. ‘edge-percentile-mean’ is the mean, over a node’s incident edges, of each edge’s percentile rank within the global distribution of all edges (in (0, 1]); it captures how high a node’s edges sit relative to all edges in the network.
modina.edge_filtering module
- modina.edge_filtering.filter(scores1, scores2, context1, context2, filter_method=None, filter_param=0.0, filter_metric=None, filter_rule=None, path=None)[source]
Filter association scores and context data based on the specified filtering configurations.
- Parameters:
scores1 (
DataFrame) – Statistical association scores of Context 1.scores2 (
DataFrame) – Statistical association scores of Context 2.context1 (
DataFrame) – The first context for the differential network analysis.context2 (
DataFrame) – The second context for the differential network analysis.filter_method (
Optional[str]) – Method used for filtering. Defaults to None.filter_param (
float) – Parameter for the specified filtering method. Defaults to 0.0.filter_metric (
Optional[str]) – Edge metric used for filtering. Options include ‘raw-P’ and ‘rescaled-E’. Defaults to None.filter_rule (
Optional[str]) – Rule to integrate the networks during filtering. Defaults to None.path (
Optional[str]) – Optional path to save the filtered scores and context data as CSV files. Defaults to None.
- Return type:
Tuple[DataFrame,DataFrame,DataFrame,DataFrame]- Returns:
A tuple containing the filtered scores and context data.
- modina.edge_filtering.filter_differential(edges_diff, edge_metric, filter_method=None, filter_param=0.0, path=None)[source]
Filter the differential network by retaining the strongest edges according to the already-computed edge metric. Because the differential network is a single graph, there is no per-context integration (‘filter_rule’) and no p-value-vs-effect-size choice (‘filter_metric’): filtering always operates on the provided edge_metric.
- Parameters:
edges_diff (
DataFrame) – Differential edge scores with columns ‘label1’, ‘label2’, ‘test_type’ and edge_metric.edge_metric (
str) – Name of the (absolute) differential edge-metric column to filter on.filter_method (
Optional[str]) – Method used for filtering (‘degree’ or ‘density’). Defaults to None.filter_param (
float) – Parameter for the specified filtering method. Defaults to 0.0.path (
Optional[str]) – Optional path to save the filtered edges and per-node statistics as CSV files. Defaults to None.
- Return type:
Tuple[DataFrame,DataFrame]- Returns:
A tuple (edges_filtered, edge_node_stats) with the filtered differential edges and the per-node statistics recomputed over the retained edges.
- modina.edge_filtering.filter_single(scores, context, filter_method=None, filter_param=0.0, filter_metric=None, path=None)[source]
Filter the association scores and context data of a single context network based on the specified filtering configurations. Unlike
filter(), this operates on one network only and therefore does not take a ‘filter_rule’ (there is nothing to integrate across contexts).- Parameters:
scores (
DataFrame) – Statistical association scores of the context.context (
DataFrame) – Observed data of the context (rows: samples, columns: variables).filter_method (
Optional[str]) – Method used for filtering (‘degree’ or ‘density’). Defaults to None.filter_param (
float) – Parameter for the specified filtering method. Defaults to 0.0.filter_metric (
Optional[str]) – Edge metric used for filtering. Options are ‘raw-P’ and ‘rescaled-E’. Defaults to None.path (
Optional[str]) – Optional path to save the filtered scores and context data as CSV files. Defaults to None.
- Return type:
Tuple[DataFrame,DataFrame]- Returns:
A tuple (scores_filtered, context_filtered) with the filtered scores and context data.
modina.pipeline module
- modina.pipeline.diffnet_analysis(context1, context2, meta_file, edge_metric=None, node_metric=None, ranking_alg='PageRank+', filter_method=None, filter_param=0.0, filter_metric=None, filter_rule=None, filter_target=None, test_type='nonparametric', nan_value=None, correction='bh', correction_scope='global', num_workers=1, n_min_valid=10, project_path=None, name1='context1', name2='context2')[source]
Wrapper function to perform an end-to-end differential network analysis following the moDiNA pipeline.
- Parameters:
context1 (
DataFrame) – Observed data of Context 1 (rows: samples, columns: variables).context2 (
DataFrame) – Observed data of Context 2 (rows: samples, columns: variables).meta_file (
DataFrame) – Metadata file containing a ‘label’ and ‘type’ column to specify the data type of each variable.test_type (
str) – Type of statistical tests to use for association score calculation. Defaults to ‘nonparametric’.nan_value (
Optional[int]) – Numerical value used for NaN values in the context data. If None, an error will be raised if such values are present. Defaults to None.correction (
str) – Correction method for multiple testing. Defaults to ‘bh’.correction_scope (
str) – Scope of the multiple-testing correction. ‘global’ pools all edges (and all nodes for STC) into one family; ‘local’ corrects separately within each statistical test type and, for STC, within each test type used to compare nodes across contexts (chi-squared for nominal/binary nodes, Mann-Whitney U for ordinal nodes, and t-test or Mann-Whitney U for continuous nodes). Defaults to ‘global’.num_workers (
int) – Number of workers for parallel processing. Defaults to 1.n_min_valid (
int) – Minimum number of samples (after pairwise removal of missing values) a pairwise association test must be based on. Edges below this in either context are removed from both. Defaults to 10.filter_method (
Optional[str]) – Method used for filtering. Defaults to None.filter_param (
float) – Parameter for the specified filtering method. Defaults to 0.0.filter_metric (
Optional[str]) – Edge metric used for filtering. Only used when filter_target is ‘context-specific’. Defaults to None.filter_rule (
Optional[str]) – Rule to integrate the networks during filtering. Only used when filter_target is ‘context-specific’. Defaults to None.filter_target (
Optional[str]) – Which network to filter. None (default) performs no filtering. ‘context-specific’ filters the two context networks before building the differential network (using filter_metric and filter_rule); ‘differential’ filters the differential network on the computed edge_metric (filter_metric and filter_rule are ignored). When filter_target is not None, filter_method must be provided.edge_metric (
Optional[str]) – Edge metric used to construct the differential network: ‘diff-<score>’ with score P, LP, RE, VE, LPxRE or LPxVE, or ‘sum-LP-VE’ / ‘sum-LP-RE’.node_metric (
Optional[str]) – Node metric used to construct the differential network: ‘<family>-<score>’ with family WDC, PRC or STC and the same scores as for edge metrics.ranking_alg (
str) – Ranking algorithm to compute. Options are ‘PageRank+’, ‘PageRank’, ‘MeanEdgeRank’, ‘MedianEdgeRank’, ‘nodeRank’, ‘MeanProductRank’ and ‘MedianProductRank’. Defaults to ‘PageRank+’.name1 (
str) – Name of Context 1. Used for saving files. Defaults to ‘context1’.name2 (
str) – Name of Context 2. Used for saving files. Defaults to ‘context2’.project_path (
Optional[str]) – Optional path to save results. Defaults to None.
- Returns:
A tuple (ranking, edges_diff, nodes_diff, config) containing the computed ranking, differential edges, differential nodes, and configuration parameters.
modina.ranking module
- modina.ranking.compute_ranking(nodes_diff, edges_diff, ranking_alg, path=None, meta_file=None, edge_node_stats=None)[source]
Compute a ranking based on the specified ranking algorithm.
The output is additionally enriched with the node-metric value (when a node metric was employed) and with per-node statistics over the incident edges (when an edge metric was employed). ‘MeanEdgeRank’ / ‘MedianEdgeRank’ score each node by the mean / median edge metric over its incident edges (‘edge-mean’ / ‘edge-median’). ‘MeanProductRank’ / ‘MedianProductRank’ rank nodes by the product of their rank by the node metric and their rank by that mean / median incident edge metric; their score is therefore better when lower, while ‘rank’ is 1 for the top node for every algorithm. Algorithms combining node and edge metrics (‘PageRank+’, ‘MeanProductRank’, ‘MedianProductRank’) require both to use the same effect-size rescaling (RE or VE); P- and LP-based metrics can be combined with either.
- Parameters:
nodes_diff (
Union[DataFrame,Series,None]) – Differential node scores.edges_diff (
Union[DataFrame,Series,None]) – Differential edge scores.ranking_alg (
str) – Ranking algorithm to compute. Options are ‘PageRank+’, ‘PageRank’, ‘MeanEdgeRank’, ‘MedianEdgeRank’, ‘nodeRank’, ‘MeanProductRank’ and ‘MedianProductRank’.meta_file (
Optional[DataFrame]) – Metadata file containing a ‘label’ and ‘type’ column to specify the data type of each variable.path (
Optional[str]) – Optional path to save the ranking as a CSV file.edge_node_stats (
Optional[DataFrame]) – Optional precomputed per-node edge statistics (see edge_node_statistics). If not provided but an edge metric was employed, the statistics are computed on the fly.
- Return type:
DataFrame|Series- Returns:
A DataFrame containing the ranking, enriched with the node-metric value and per-node edge statistics.
modina.statistics_utils module
- modina.statistics_utils.find_flagged_variables(scores1, scores2)[source]
Return the sorted list of variables that appear as an edge endpoint in only one of scores1/scores2 – i.e. variables flagged (single observed category, or entirely missing) in exactly one context and therefore absent from that context’s association scores.
- Parameters:
scores1 (
DataFrame) – Association scores of Context 1.scores2 (
DataFrame) – Association scores of Context 2.
- Return type:
list- Returns:
Sorted list of flagged variable names.
- modina.statistics_utils.reconcile_flagged_variables(scores1, scores2, context1=None, context2=None, path=None)[source]
Align two context-specific networks onto their common set of edges, so scores1 and scores2 stay row-aligned, and report which variables ended up with no edges left in either context.
- Parameters:
scores1 (
DataFrame) – Association scores of Context 1.scores2 (
DataFrame) – Association scores of Context 2.context1 (
Optional[DataFrame]) – Optional observed data of Context 1; if given, variables with no edges left are also dropped as columns.context2 (
Optional[DataFrame]) – Optional observed data of Context 2; if given, variables with no edges left are also dropped as columns.path (
Optional[str]) – Optional path to save the list of removed variables as a CSV file. Defaults to None.
- Return type:
Tuple[DataFrame,DataFrame,Optional[DataFrame],Optional[DataFrame],list]- Returns:
A tuple (scores1, scores2, context1, context2, removed_variables), with scores1/scores2 restricted to their common edges (row-aligned) and context1/context2 returned unchanged (None) if not provided. ‘removed_variables’ is the sorted list of variables that no longer appear as an edge endpoint in either context (empty if none) – callers should attach it as .attrs[‘removed_variables’] on whatever they return, so it stays discoverable to a caller even when no ‘path’ is given to write it to disk (e.g. a direct Python/API call that never passes through a Nextflow bin script).
Module contents
- modina.check_rescaling_consistency(node_metric, edge_metric, ranking_alg=None)[source]
Node and edge metrics combined in one ranking must not mix the RE and VE effect-size rescalings.
- Return type:
None
- modina.compute_context_scores(context_data, meta_file, test_type='nonparametric', correction='bh', num_workers=1, path=None, nan_value=None, name='context1', n_min_valid=10, correction_scope='global')[source]
Compute association scores for a given context.
- Parameters:
context_data (
DataFrame) – Raw context data (rows: samples, columns: variables).meta_file (
DataFrame) – Metadata file containing a ‘label’ and ‘type’ column to specify the data type of each variable.test_type (
str) – Type of tests to use for network inference. Defaults to ‘nonparametric’.correction (
str) – Correction method for multiple testing. Defaults to ‘bh’.num_workers (
int) – Number of workers for parallel processing. Defaults to 1.path (
Optional[str]) – Optional path to save the computed scores as a CSV file. Defaults to None.nan_value (
Optional[float]) – Numerical value used for NaN values in the context data. If None, an error will be raised if such values are present. Defaults to None.name (
str) – Name of the context. Used for saving files. Defaults to ‘context’.n_min_valid (
int) – Minimum number of samples (after pairwise removal of missing values) a pairwise test must be based on. Pairs below this are excluded from this context’s network (and, during differential analysis, from the other context’s as well). Defaults to 10.correction_scope (
str) – Scope of the multiple-testing correction. ‘global’ pools all edges into one correction family; ‘local’ corrects separately within each statistical test type (e.g. Pearson, Spearman, t-test, Mann-Whitney U, chi-squared, Kruskal-Wallis). Defaults to ‘global’.
- Return type:
DataFrame- Returns:
A pd.DataFrame containing the computed association scores.
- modina.compute_diff_edges(scores1, scores2, edge_metric, path=None, name1='context1', name2='context2')[source]
Compute differential edge scores based on the specified edge metric.
Every edge metric is built from per-context scores (see statistics_utils.add_score_columns): ‘diff-<score>’ is the absolute between-context difference of one score (P, LP, RE, VE, LPxRE or LPxVE), and ‘sum-LP-VE’ / ‘sum-LP-RE’ add up the absolute differences of LP and VE / RE. ‘diff-Z’ is the change of the standardised effect size z / sqrt(n) in units of its standard error (see _diff_z); it needs the per-edge sample size column ‘n’ in both score tables.
- Parameters:
scores1 (
DataFrame) – Statistical association scores of Context 1, rescaled and potentially filtered.scores2 (
DataFrame) – Statistical association scores of Context 2, rescaled and potentially filtered.edge_metric (
str) – Edge metric to compute the differential edge scores.path (
Optional[str]) – Optional path to save the differential edge scores as a CSV file. Defaults to None.name1 (
str) – Name of Context 1, used to label the per-contextraw-P/raw-Ecolumns. Defaults to ‘context1’.name2 (
str) – Name of Context 2, used to label the per-contextraw-P/raw-Ecolumns. Defaults to ‘context2’.
- Return type:
Tuple[DataFrame,DataFrame]- Returns:
A tuple (edges_diff, edge_node_stats): the computed differential edge scores and a node-indexed DataFrame of per-node statistics over incident edges (see edge_node_statistics).
- modina.compute_diff_network(scores1, scores2, context1, context2, edge_metric=None, node_metric=None, correction='bh', correction_scope='global', num_workers=1, path=None, format='csv', meta_file=None, test_type='nonparametric', nan_value=None, name1='context1', name2='context2')[source]
Computation of a differential network defined by a node metric and an edge metric.
- Parameters:
scores1 (
DataFrame) – Statistical association scores of Context 1, rescaled and potentially filtered.scores2 (
DataFrame) – Statistical association scores of Context 2, rescaled and potentially filtered.context1 (
DataFrame) – Observed data of Context 1, potentially filtered.context2 (
DataFrame) – Observed data of Context 2, potentially filtered.edge_metric (
Optional[str]) – Edge metric used to construct the differential network.node_metric (
Optional[str]) – Node metric used to construct the differential network.correction (
str) – Correction method for multiple testing. Defaults to ‘bh’.correction_scope (
str) – Scope of the multiple-testing correction for STC: ‘global’ (all nodes in one family) or ‘local’ (nodes corrected separately per test type used to compare them across contexts). Defaults to ‘global’.num_workers (
int) – Number of workers for parallel computation of STC. Defaults to 1.path (
Optional[str]) – Optional path to save the differential scores as CSV files. Defaults to None.format (
str) – File format to save the differential network. Options are ‘csv’ and ‘graphml’. Defaults to ‘csv’.meta_file (
Optional[DataFrame]) – Meta file containing the node types. Only needed for the STC node metrics. Defaults to None.test_type (
str) – Test type to use for continuous nodes in the STC node metrics. Defaults to ‘nonparametric’.nan_value (
Optional[int]) – Numerical value used for NaN values in the context data. If None, an error will be raised if such values are present. Defaults to None.
- Return type:
Tuple[DataFrame|None,DataFrame|None,DataFrame|None]- Returns:
A tuple (edges_diff, nodes_diff, edge_node_stats) containing the computed differential edges, differential nodes, and per-node statistics over incident edges (None if no edge_metric).
- modina.compute_diff_nodes(scores1, scores2, context1, context2, node_metric, correction='bh', meta_file=None, test_type='nonparametric', nan_value=None, num_workers=1, path=None, correction_scope='global')[source]
Compute differential node scores based on the specified node metric.
Node metrics are ‘<family>-<score>’ with family WDC (weighted degree centrality), PRC (PageRank centrality) or STC (statistical test centrality) and score P, LP, RE, VE, LPxRE or LPxVE (see statistics_utils.add_score_columns). WDC and PRC use the score as edge weight in each context and take the absolute difference of the max-normalised centralities; STC tests each node between the contexts and derives the score from the test’s adjusted p-value and effect size.
- Parameters:
scores1 (
DataFrame) – Statistical association scores of Context 1, rescaled and potentially filtered.scores2 (
DataFrame) – Statistical association scores of Context 2, rescaled and potentially filtered.context1 (
DataFrame) – Observed data of Context 1, potentially filtered.context2 (
DataFrame) – Observed data of Context 2, potentially filtered.node_metric (
str) – Node metric to compute the differential node scores.correction (
str) – Correction method for multiple testing. Only needed for the STC node metrics. Defaults to ‘bh’.meta_file (
Optional[DataFrame]) – Meta file containing the node types. Only needed for the STC node metrics. Defaults to None.test_type (
str) – Test type to compare continuous variables across contexts for the STC node metrics. Defaults to ‘nonparametric’.nan_value (
Optional[int]) – Numerical value used for NaN values in the context data. If None, an error will be raised if such values are present. Defaults to None.num_workers (
int) – Number of workers for parallel computation of STC. Only needed for the STC node metrics. Defaults to 1.path (
Optional[str]) – Optional path to save the differential node scores as a CSV file. Defaults to None.correction_scope (
str) – Scope of the multiple-testing correction for STC: ‘global’ (all nodes in one family) or ‘local’ (nodes corrected separately per test type used to compare them across contexts). Defaults to ‘global’.
- Return type:
DataFrame- Returns:
A DataFrame containing the computed differential node scores.
- modina.compute_ranking(nodes_diff, edges_diff, ranking_alg, path=None, meta_file=None, edge_node_stats=None)[source]
Compute a ranking based on the specified ranking algorithm.
The output is additionally enriched with the node-metric value (when a node metric was employed) and with per-node statistics over the incident edges (when an edge metric was employed). ‘MeanEdgeRank’ / ‘MedianEdgeRank’ score each node by the mean / median edge metric over its incident edges (‘edge-mean’ / ‘edge-median’). ‘MeanProductRank’ / ‘MedianProductRank’ rank nodes by the product of their rank by the node metric and their rank by that mean / median incident edge metric; their score is therefore better when lower, while ‘rank’ is 1 for the top node for every algorithm. Algorithms combining node and edge metrics (‘PageRank+’, ‘MeanProductRank’, ‘MedianProductRank’) require both to use the same effect-size rescaling (RE or VE); P- and LP-based metrics can be combined with either.
- Parameters:
nodes_diff (
Union[DataFrame,Series,None]) – Differential node scores.edges_diff (
Union[DataFrame,Series,None]) – Differential edge scores.ranking_alg (
str) – Ranking algorithm to compute. Options are ‘PageRank+’, ‘PageRank’, ‘MeanEdgeRank’, ‘MedianEdgeRank’, ‘nodeRank’, ‘MeanProductRank’ and ‘MedianProductRank’.meta_file (
Optional[DataFrame]) – Metadata file containing a ‘label’ and ‘type’ column to specify the data type of each variable.path (
Optional[str]) – Optional path to save the ranking as a CSV file.edge_node_stats (
Optional[DataFrame]) – Optional precomputed per-node edge statistics (see edge_node_statistics). If not provided but an edge metric was employed, the statistics are computed on the fly.
- Return type:
DataFrame|Series- Returns:
A DataFrame containing the ranking, enriched with the node-metric value and per-node edge statistics.
- modina.diffnet_analysis(context1, context2, meta_file, edge_metric=None, node_metric=None, ranking_alg='PageRank+', filter_method=None, filter_param=0.0, filter_metric=None, filter_rule=None, filter_target=None, test_type='nonparametric', nan_value=None, correction='bh', correction_scope='global', num_workers=1, n_min_valid=10, project_path=None, name1='context1', name2='context2')[source]
Wrapper function to perform an end-to-end differential network analysis following the moDiNA pipeline.
- Parameters:
context1 (
DataFrame) – Observed data of Context 1 (rows: samples, columns: variables).context2 (
DataFrame) – Observed data of Context 2 (rows: samples, columns: variables).meta_file (
DataFrame) – Metadata file containing a ‘label’ and ‘type’ column to specify the data type of each variable.test_type (
str) – Type of statistical tests to use for association score calculation. Defaults to ‘nonparametric’.nan_value (
Optional[int]) – Numerical value used for NaN values in the context data. If None, an error will be raised if such values are present. Defaults to None.correction (
str) – Correction method for multiple testing. Defaults to ‘bh’.correction_scope (
str) – Scope of the multiple-testing correction. ‘global’ pools all edges (and all nodes for STC) into one family; ‘local’ corrects separately within each statistical test type and, for STC, within each test type used to compare nodes across contexts (chi-squared for nominal/binary nodes, Mann-Whitney U for ordinal nodes, and t-test or Mann-Whitney U for continuous nodes). Defaults to ‘global’.num_workers (
int) – Number of workers for parallel processing. Defaults to 1.n_min_valid (
int) – Minimum number of samples (after pairwise removal of missing values) a pairwise association test must be based on. Edges below this in either context are removed from both. Defaults to 10.filter_method (
Optional[str]) – Method used for filtering. Defaults to None.filter_param (
float) – Parameter for the specified filtering method. Defaults to 0.0.filter_metric (
Optional[str]) – Edge metric used for filtering. Only used when filter_target is ‘context-specific’. Defaults to None.filter_rule (
Optional[str]) – Rule to integrate the networks during filtering. Only used when filter_target is ‘context-specific’. Defaults to None.filter_target (
Optional[str]) – Which network to filter. None (default) performs no filtering. ‘context-specific’ filters the two context networks before building the differential network (using filter_metric and filter_rule); ‘differential’ filters the differential network on the computed edge_metric (filter_metric and filter_rule are ignored). When filter_target is not None, filter_method must be provided.edge_metric (
Optional[str]) – Edge metric used to construct the differential network: ‘diff-<score>’ with score P, LP, RE, VE, LPxRE or LPxVE, or ‘sum-LP-VE’ / ‘sum-LP-RE’.node_metric (
Optional[str]) – Node metric used to construct the differential network: ‘<family>-<score>’ with family WDC, PRC or STC and the same scores as for edge metrics.ranking_alg (
str) – Ranking algorithm to compute. Options are ‘PageRank+’, ‘PageRank’, ‘MeanEdgeRank’, ‘MedianEdgeRank’, ‘nodeRank’, ‘MeanProductRank’ and ‘MedianProductRank’. Defaults to ‘PageRank+’.name1 (
str) – Name of Context 1. Used for saving files. Defaults to ‘context1’.name2 (
str) – Name of Context 2. Used for saving files. Defaults to ‘context2’.project_path (
Optional[str]) – Optional path to save results. Defaults to None.
- Returns:
A tuple (ranking, edges_diff, nodes_diff, config) containing the computed ranking, differential edges, differential nodes, and configuration parameters.
- modina.edge_node_statistics(edges_diff, edge_metric)[source]
Compute summary statistics of the (absolute) edge metric over the edges incident to each node.
- Parameters:
edges_diff (
DataFrame) – Differential edge scores with columns ‘label1’, ‘label2’ and edge_metric.edge_metric (
str) – Name of the (absolute) edge metric column.
- Return type:
DataFrame- Returns:
A node-indexed DataFrame with columns ‘edge-min’, ‘edge-max’, ‘edge-median’, ‘edge-mean’, ‘edge-sd’ and ‘edge-percentile-mean’. sd is the sample standard deviation (ddof=1) and is NaN for nodes with a single incident edge. ‘edge-percentile-mean’ is the mean, over a node’s incident edges, of each edge’s percentile rank within the global distribution of all edges (in (0, 1]); it captures how high a node’s edges sit relative to all edges in the network.
- modina.effect_rescaling(metric)[source]
Effect-size rescaling a metric uses: ‘RE’ (rank-based), ‘VE’ (variance-explained) or None (P, LP).
- Return type:
Optional[str]
- modina.filter(scores1, scores2, context1, context2, filter_method=None, filter_param=0.0, filter_metric=None, filter_rule=None, path=None)[source]
Filter association scores and context data based on the specified filtering configurations.
- Parameters:
scores1 (
DataFrame) – Statistical association scores of Context 1.scores2 (
DataFrame) – Statistical association scores of Context 2.context1 (
DataFrame) – The first context for the differential network analysis.context2 (
DataFrame) – The second context for the differential network analysis.filter_method (
Optional[str]) – Method used for filtering. Defaults to None.filter_param (
float) – Parameter for the specified filtering method. Defaults to 0.0.filter_metric (
Optional[str]) – Edge metric used for filtering. Options include ‘raw-P’ and ‘rescaled-E’. Defaults to None.filter_rule (
Optional[str]) – Rule to integrate the networks during filtering. Defaults to None.path (
Optional[str]) – Optional path to save the filtered scores and context data as CSV files. Defaults to None.
- Return type:
Tuple[DataFrame,DataFrame,DataFrame,DataFrame]- Returns:
A tuple containing the filtered scores and context data.
- modina.filter_differential(edges_diff, edge_metric, filter_method=None, filter_param=0.0, path=None)[source]
Filter the differential network by retaining the strongest edges according to the already-computed edge metric. Because the differential network is a single graph, there is no per-context integration (‘filter_rule’) and no p-value-vs-effect-size choice (‘filter_metric’): filtering always operates on the provided edge_metric.
- Parameters:
edges_diff (
DataFrame) – Differential edge scores with columns ‘label1’, ‘label2’, ‘test_type’ and edge_metric.edge_metric (
str) – Name of the (absolute) differential edge-metric column to filter on.filter_method (
Optional[str]) – Method used for filtering (‘degree’ or ‘density’). Defaults to None.filter_param (
float) – Parameter for the specified filtering method. Defaults to 0.0.path (
Optional[str]) – Optional path to save the filtered edges and per-node statistics as CSV files. Defaults to None.
- Return type:
Tuple[DataFrame,DataFrame]- Returns:
A tuple (edges_filtered, edge_node_stats) with the filtered differential edges and the per-node statistics recomputed over the retained edges.
- modina.filter_single(scores, context, filter_method=None, filter_param=0.0, filter_metric=None, path=None)[source]
Filter the association scores and context data of a single context network based on the specified filtering configurations. Unlike
filter(), this operates on one network only and therefore does not take a ‘filter_rule’ (there is nothing to integrate across contexts).- Parameters:
scores (
DataFrame) – Statistical association scores of the context.context (
DataFrame) – Observed data of the context (rows: samples, columns: variables).filter_method (
Optional[str]) – Method used for filtering (‘degree’ or ‘density’). Defaults to None.filter_param (
float) – Parameter for the specified filtering method. Defaults to 0.0.filter_metric (
Optional[str]) – Edge metric used for filtering. Options are ‘raw-P’ and ‘rescaled-E’. Defaults to None.path (
Optional[str]) – Optional path to save the filtered scores and context data as CSV files. Defaults to None.
- Return type:
Tuple[DataFrame,DataFrame]- Returns:
A tuple (scores_filtered, context_filtered) with the filtered scores and context data.
- modina.find_flagged_variables(scores1, scores2)[source]
Return the sorted list of variables that appear as an edge endpoint in only one of scores1/scores2 – i.e. variables flagged (single observed category, or entirely missing) in exactly one context and therefore absent from that context’s association scores.
- Parameters:
scores1 (
DataFrame) – Association scores of Context 1.scores2 (
DataFrame) – Association scores of Context 2.
- Return type:
list- Returns:
Sorted list of flagged variable names.
- modina.save_gt(groundtruths, path, mode='node', edge_magnitude=None, node_stats=None, node_shift_magnitude=None)[source]
Write ground truth nodes or edges to a two- (or, with the optional data below, wider-) column file.
- Parameters:
groundtruths – (shift_nodes, corr_nodes, shift_corr_nodes) as returned by simulate_copula.
path – File to write.
mode – ‘node’ writes one row per (node, description); ‘edge’ writes one row per (edge, description), edge = ‘_’.join(sorted(pair)).
edge_magnitude – Optional, mode=’edge’ only. Dict keyed by frozenset({node1, node2}) -> the correlation magnitude injected on that edge. When given, an extra corr column is appended. Omit to reproduce the historical two-column file exactly.
node_stats – Optional, mode=’node’ only. Dict node -> (n_edges_tweaked, sum_corr). When given, two extra columns are appended; a node absent from the dict (e.g. mean-shift-only) is written as (0, 0.0). Omit to reproduce the historical two-column file exactly.
node_shift_magnitude – Optional, mode=’node’ only. Dict node -> mean-shift magnitude injected on that node (e.g. effects[‘node_shift_magnitude’]). When given, an extra shift column is appended; a node with no mean-shift draw of its own (e.g. correlation-only) is written as 0.0. Omit to reproduce the file exactly as it was before this column existed.
The magnitude columns are named shift and corr after the quantity they hold, matching the shift_low/shift_high and corr_low/corr_high parameters the values are drawn from.
- modina.simulate_copula(path=None, name1='context1', name2='context2', n_bi=50, n_cont=50, n_cat=50, n_samples_1=500, n_samples_2=500, n_shift_cont=0, n_shift_bi=0, n_shift_cat=0, n_corr_cont_cont=0, n_corr_bi_bi=0, n_corr_cat_cat=0, n_corr_bi_cont=0, n_corr_bi_cat=0, n_corr_cont_cat=0, n_both_cont_cont=0, n_both_bi_bi=0, n_both_cat_cat=0, n_both_bi_cont=0, n_both_bi_cat=0, n_both_cont_cat=0, *, corr_low, corr_high, shift_low, shift_high, hub_reuse_prob=0.0, background_strength=0.2, background_density=2, background_seed=None, background_model='modules', module_size=None, module_fraction=0.1, module_correlation=0.2)[source]
Simulate two contexts with binary and continuous nodes using a Gaussian copula.
- Parameters:
path – Path to save the simulated contexts, the meta file and the ground truth information. If None, files are not saved.
name1 – Name of the first context.
name2 – Name of the second context.
n_bi – Number of binary nodes to simulate.
n_cont – Number of continuous nodes to simulate.
n_cat – Number of categorical nodes to simulate.
n_samples_1 – Number of samples for the first context.
n_samples_2 – Number of samples for the second context.
n_shift_cont – Number of continuous nodes with an artificially introduced mean shift.
n_shift_bi – Number of binary nodes with an artificially introduced mean shift.
n_shift_cat – Number of categorical nodes with an artificially introduced mean shift.
n_corr_cont_cont – Number of continuous node pairs with an artifically introduced correlation difference.
n_corr_bi_bi – Number of binary node pairs with an artificially introduced correlation difference.
n_corr_cat_cat – Number of categorical node pairs with an artificially introduced correlation difference.
n_corr_bi_cat – Number of binary-categorical node pairs with an artificially introduced correlation difference.
n_corr_cont_cat – Number of continuous-categorical node pairs with an artificially introduced correlation difference.
n_corr_bi_cont – Number of mixed node pairs with an artificially introduced correlation difference.
n_both_cont_cont – Number of continuous node pairs with both an aritificially introduced mean shift and correlation difference.
n_both_bi_bi – Number of binary node pairs with both an artificially introduced mean shift and correlation difference.
n_both_cat_cat – Number of categorical node pairs with both an artificially introduced mean shift and correlation difference.
n_both_bi_cat – Number of binary-categorical node pairs with both an artificially introduced mean shift and correlation difference.
n_both_cont_cat – Number of continuous-categorical node pairs with both an artificially introduced mean shift and correlation difference.
n_both_bi_cont – Number of mixed node pairs with both an artificially introduced mean shift and correlation difference.
corr_low – Lower bound of the correlation-difference magnitude range. Each correlated pair independently draws its actual magnitude uniformly from [corr_low, corr_high], so ground truth effects span a continuum – rankable by realized magnitude – rather than clustering at one fixed value. Required.
corr_high – Upper bound of the correlation-difference magnitude range. Must satisfy corr_low <= corr_high <= 0.95 (0.95 is the historical ceiling on a single edge’s magnitude). Required.
shift_low – Lower bound of the mean-shift magnitude range. Each shifted node independently draws its actual magnitude uniformly from [shift_low, shift_high], for the same reason as corr_low/corr_high. Required.
shift_high – Upper bound of the mean-shift magnitude range. Must satisfy shift_low <= shift_high. Required.
hub_reuse_prob – Fraction of each pair type’s correlation/both pairs that reuse an already-placed node as one endpoint (“hub”) instead of drawing two fresh nodes. Real differential networks concentrate rewiring on a few hub nodes rather than a perfect matching. The count per pair type is fixed at round(hub_reuse_prob * n_pairs) rather than drawn per edge, and cross-type reuses are spread so every node type loses the same share of its endpoints to hubs (see _plan_hub_reuse); this keeps the number of ground truth nodes per type consistent across replicates and types. A reused hub is picked at random among the eligible nodes of its type whose running sum(magnitude**2) stays below the background’s lambda_min**2. A reuse only ever fills one slot of a pair; the other is always a fresh node, and that fresh node – plus the hub’s original first partner, the first time the hub is reused – is permanently barred from ever being reused itself. This keeps every hub isolated from every other hub, which is what makes the sum(magnitude**2) check exact. If no eligible hub exists yet, the reuse is deferred to the next fresh pair of the same pair type. Default 0.0 gives one edge per node.
The background correlation structure is built ONCE and shared by both contexts, so every pair that is not planted has bit-identical correlation in the two contexts and the negatives stay exactly null. By default (background_model=’modules’) it consists of correlated modules (see module_fraction). The alternative, background_model=’scale_free’, is a sparse scale-free network (preferential attachment, producing a few high-degree hubs and a heavy-tailed degree distribution), turned into a dense correlation matrix (see background_strength and background_density). :type background_strength: :param background_strength: Only used with background_model=’scale_free’. Target mean |r| over
the directly-wired pairs of the background. This is a property of the resulting correlation matrix, not a value written anywhere: the diagonal loading that attains it is solved for by bisection. It is bounded above by the topology (denser wiring forces weaker correlations); an unreachable request raises with the attainable ceiling.
- Parameters:
background_density – Only used with background_model=’scale_free’. Number of connections each node makes when it is added to the network. Sets both the edge count and how large the hubs grow. Higher density lowers the reachable strength.
background_seed – Seed for the background’s own random generator. Kept separate from the global streams the rest of the simulation draws from, so changing the background does not change which pairs get planted or with what magnitudes.
background_model – ‘modules’ (default) builds a dense background of correlated modules (see _build_module_background) from module_fraction (or module_size) and module_correlation, and ignores the two scale-free parameters. ‘scale_free’ builds the scale-free background from background_strength and background_density. The module background reaches the association density of real clinical cohorts, which the scale-free one cannot: its diagonally dominant precision matrix caps the share of pairs with |r| > 0.15 at about 1%.
module_size – Number of variables per module (modules have near-equal sizes). Default None: set from module_fraction. Only used with background_model=’modules’.
module_fraction – Module size as a share of all variables, in (0, 1], used when module_size is None: module_size = max(2, round(module_fraction * n_vars)). Scaling the modules with the number of variables keeps the background density constant across variable counts. Default 0.1. Only used with background_model=’modules’.
module_correlation – Correlation between the module factors, in [0, 1). Sets the weak associations between modules and with them how dense the background is. Default 0.2. Only used with background_model=’modules’.
- Returns:
A tuple containing the two simulated contexts, a meta file, a list of ground truth nodes and a dict of per-edge/per-node injected effect sizes. - context1: pd.DataFrame of the first simulated context. - context2: pd.DataFrame of the second simulated context. - meta: pd.DataFrame containing the data type for each simulated variable. - ground_truth: A tuple containing three lists of ground truth nodes: (shift_nodes, corr_nodes, shift_corr_nodes). - effects: dict with ‘edge_magnitude’ (dict keyed by frozenset({node1, node2}) -> the
jittered correlation magnitude actually injected on that edge), ‘node_degree’ (dict node -> number of correlation edges touching it) and ‘node_sum_magnitude’ (dict node -> sum of jittered magnitudes over those edges). Feeds the extra columns written to ground_truth_edges.txt/ground_truth_nodes.txt when path is given, and lets a caller that writes those files itself (as the Nextflow CLI wrapper does) do the same. Also carries ‘node_shift_magnitude’ (dict node -> its drawn mean-shift magnitude, 0.0 for nodes with no mean-shift draw of their own), ‘corr_base’ (the shared background correlation matrix), ‘background_edges’ (the directly-wired pairs, as node-name frozensets) and ‘background_lambda_min’ (the perturbation budget the planting step was checked against).