NOVA: package-level imports and global variables

Description

Internal imports used across the package and a list of non-standard evaluation (NSE) column names suppressed for R CMD check.

Package-level imports

Description

Package-level imports

Aggregate Data by Groups

Description

Aggregates values within groups using specified method

Usage

aggregate_data(data, group_col, variable_column, value_column, method)

Arguments

Value

Aggregated data frame

Examples

test_data <- data.frame(
  Group = rep(c("A", "B"), each = 10),
  Variable = rep(paste0("V", 1:5), 4),
  Value = rnorm(20)
)
agg <- aggregate_data(test_data, "Group", "Variable", "Value", "mean")

Analyze and Visualize PCA Variable Importance

Description

This function performs comprehensive analysis of variable importance in Principal Component Analysis, generating multiple visualization types including loading biplots, importance rankings, PC comparisons, and heatmaps. It extracts variable contributions to specified principal components and creates publication-ready plots with detailed statistical summaries.

Usage

analyze_pca_variable_importance_general(
  pca_result = NULL,
  output_dir = tempdir(),
  experiment_name = "PCA_Analysis",
  pc_x = "PC1",
  pc_y = "PC2",
  color_scheme = "default",
  top_n = 15,
  min_loading_threshold = 0.1,
  save_plots = TRUE,
  show_labels = TRUE,
  verbose = TRUE
)

Arguments

Details

The function calculates multiple importance metrics for each variable:

• PC loadings: Direct loading values for specified principal components

• Combined importance: Euclidean distance combining both PC loadings

• Contribution percentages: Percent contribution to each PC's total variance

• Ranking: Variables ranked by combined importance score

Four visualization types are generated:

• Loading Biplot: Scatter plot showing variable loadings on both PCs with size indicating importance

• Importance Bar Chart: Ranked bar chart of top variables by combined importance

• PC Comparison: Side-by-side comparison of absolute loadings for both PCs

• Loading Heatmap: Color-coded matrix showing loading values and directions

The function automatically:

• Validates input PCA objects from various sources

• Calculates variance explained by each principal component

• Creates publication-ready plots with consistent theming

• Exports detailed CSV files with variable rankings and analysis summaries

• Provides comprehensive statistical summaries

Color schemes provide different aesthetic options:

• default: Blue/red palette suitable for most publications

• viridis: Colorblind-friendly viridis color scale

• colorbrewer: ColorBrewer palettes optimized for scientific visualization

View top variables using head(results$selected_variables)

Value

A list containing:

plots

Named list of ggplot objects: 'biplot', 'importance_bar', 'pc_comparison', 'heatmap'

variable_importance

Data frame with comprehensive variable importance metrics for all variables

selected_variables

Data frame containing the top N most important variables with detailed statistics

analysis_summary

List with key analysis metrics and variance explained information

config_used

List documenting all parameters used in the analysis

Output Files

When save_plots = TRUE, the function creates files in the specified output directory (default: "pca_plots"). For CRAN compliance, use tempdir() for the output directory:

• PNG files for each visualization type

• CSV file with complete variable importance rankings

• CSV file with selected top variables and detailed metrics

• CSV file with analysis summary and metadata

See Also

prcomp for PCA computation, biplot for basic PCA plotting

Apply Enhanced Scaling Methods

Description

Applies various scaling methods to matrix data for heatmap visualization

Usage

apply_scaling_enhanced(matrix_data, scale_method, verbose = FALSE)

Arguments

Value

Scaled matrix

Clean Heatmap Matrix

Description

Removes rows and columns with insufficient finite values from matrix

Usage

clean_heatmap_matrix(matrix_data, min_finite = 2, verbose = FALSE)

Arguments

Value

Cleaned matrix or NULL if insufficient data

Create Enhanced Annotations for Heatmaps

Description

Creates annotation data frames and color schemes for heatmap visualization

Usage

create_annotations_enhanced(rownames_vector, factor_cols)

Arguments

Value

List containing annotations data frame and color schemes

Create Enhanced Color Palettes

Description

Creates color palettes and breaks for heatmap visualization

Usage

create_color_palette_enhanced(
  palette_name = "yellow_purple",
  custom_colors = NULL,
  data_matrix = NULL
)

Arguments

Value

List containing colors and breaks

Create Enhanced Heatmaps for Multi-Electrode Array (MEA) Data Analysis

Description

This function generates comprehensive heatmap visualizations for MEA data analysis, including individual grouping variable heatmaps, combined interaction heatmaps, and variable correlation matrices. It provides flexible scaling, clustering, and customization options with automatic quality filtering and missing data handling.

Usage

create_mea_heatmaps_enhanced(
  data = NULL,
  processing_result = NULL,
  config = NULL,
  value_column = "Normalized_Value",
  variable_column = "Variable",
  grouping_columns = c("Treatment", "Genotype"),
  sample_id_columns = c("Well"),
  timepoint_column = "Timepoint",
  scale_method = "z_score",
  aggregation_method = "mean",
  missing_value_handling = "remove",
  cluster_method = "euclidean",
  cluster_rows = TRUE,
  cluster_cols = TRUE,
  create_individual_heatmaps = TRUE,
  create_combined_heatmap = TRUE,
  create_variable_correlation = TRUE,
  output_dir = NULL,
  save_plots = FALSE,
  plot_format = "png",
  plot_width = 10,
  plot_height = 8,
  dpi = 300,
  fontsize = 10,
  angle_col = 45,
  show_rownames = TRUE,
  show_colnames = TRUE,
  return_data = TRUE,
  verbose = TRUE,
  quality_threshold = 0.8,
  min_observations = 3
)

Arguments

Details

The function performs several key operations:

• Quality filtering: Removes variables with insufficient data completeness

• Missing value handling: Multiple strategies for dealing with NA values

• Data aggregation: Combines multiple measurements per group using specified method

• Scaling: Applies normalization methods appropriate for heatmap visualization

• Clustering: Hierarchical clustering of rows and/or columns using specified distance metrics

• Visualization: Creates publication-ready heatmaps with proper color schemes and annotations

For scaling methods:

• z_score: Centers data around mean with unit variance (best for comparing relative changes)

• min_max: Scales to 0-1 range (best for absolute comparisons)

• robust: Uses median and MAD for outlier-resistant scaling

• none: No scaling applied

The function automatically adjusts plot dimensions based on data size and uses optimized color palettes appropriate for the scaling method chosen (diverging palettes for z_score/robust, sequential palettes for min_max).

Value

A list containing:

individual_heatmaps

Named list of heatmap objects for each grouping variable

combined_heatmap

Heatmap object for grouping variable interactions (if applicable)

variable_correlation

List with correlation heatmap and correlation matrix

metadata

List containing processing information and parameters used

Each heatmap object contains: heatmap (pheatmap object), scaled_data (processed matrix), raw_data (aggregated input data), annotation (row annotations), annotation_colors (color schemes), and scaling_info (scaling parameters).

Discover MEA Data Structure

Description

This function scans a directory containing MEA (Multi-Electrode Array) experiment folders and analyzes the structure of CSV files to identify experiments, timepoints, measured variables, treatments, and genotypes. It provides a comprehensive overview of the data organization without loading all files into memory.

Usage

discover_mea_structure(
  main_dir,
  experiment_pattern = "MEA\\d+",
  file_pattern = "\\.csv$",
  verbose = TRUE
)

Arguments

Details

The function expects MEA CSV files with standard format: - Row 121: Well identifiers (A1, A2, B1, etc.) - Row 122: Treatment conditions - Row 123: Genotype information - Row 124: Exclusion flags - Rows 125-168: Variable names and measurements

Discover structure of MEA data (requires data directory)

Value

A list containing: - experiments: List of experiment info (directories, files, timepoints, metadata) - all_timepoints: Vector of all unique timepoints found across experiments - all_variables: Vector of all unique measured variables - potential_baselines: Timepoints that might serve as baseline conditions - experiment_count: Total number of experiments found - discovery_timestamp: When the analysis was performed

Handle Missing Values in MEA Data

Description

Handles missing values in MEA datasets using various imputation strategies or removal methods.

Usage

handle_missing_values(data, value_column, method, verbose)

Arguments

Value

Data frame with missing values handled according to specified method

Examples

test_data <- data.frame(
  ID = 1:10,
  Value = c(1.2, NA, 3.4, 2.1, NA, 5.6, 4.3, NA, 2.8, 3.9)
)
cleaned <- handle_missing_values(test_data, "Value", "remove", FALSE)

Null Coalescing Operator

Description

Returns the left-hand side if not NULL, otherwise the right-hand side

Usage

null_coalesce(lhs, rhs)

Arguments

Value

lhs if not NULL, otherwise rhs

Examples

null_coalesce(5, 10)
null_coalesce(NULL, 10)

Enhanced PCA Analysis for MEA Data

Description

This function performs Principal Component Analysis (PCA) on MEA data with extensive flexibility for data input sources, parameter configuration, and output options. It handles missing values, applies variance filtering, creates visualization plots, and provides comprehensive results suitable for downstream analysis.

Usage

pca_analysis_enhanced(
  normalized_data = NULL,
  data_path = NULL,
  config = NULL,
  processing_result = NULL,
  min_var = NULL,
  impute = NULL,
  scale_data = NULL,
  n_components = NULL,
  variance_cutoff = NULL,
  grouping_variables = NULL,
  sample_id_components = NULL,
  value_column = "Normalized_Value",
  variable_column = "Variable",
  timepoint_column = "Timepoint",
  output_path = NULL,
  verbose = TRUE
)

Arguments

Details

The function provides three flexible data input methods: 1. **processing_result**: Direct output from process_mea_flexible function 2. **data_path**: Path to Excel file with normalized_data sheet 3. **normalized_data**: Pre-loaded data frame in long format

Data processing includes: - Automatic detection of available columns - Flexible sample ID creation from specified components - Missing value imputation (mean, median, or zero) - Variance-based variable filtering - Automatic scaling option - Creation of elbow plot for component selection

The function handles common MEA data challenges: - Missing timepoint or treatment information - Inconsistent column naming - Mixed data types and missing values - Variable numbers of experiments and conditions

Method 1: Use output from MEA processing function process_mea_flexible("/path/to/data", baseline_timepoint = "baseline") pca_analysis_enhanced(processing_result = mea_result)

Method 2: Load from saved Excel file pca_analysis_enhanced(data_path = "/path/to/processed_data.xlsx")

Method 3: Use pre-loaded data with custom parameters normalized_data = my_data

Value

A list containing: - pca_result: Complete prcomp() object with PCA results - plot_data: Data frame ready for plotting with PC scores and metadata - variance_explained: Vector of variance explained by each component - cumulative_variance: Vector of cumulative variance explained - elbow_plot: ggplot2 object showing variance explained by components - elbow_data: Data frame underlying the elbow plot - components_needed: Number of components needed for various variance thresholds - count_summary: Summary of sample counts by groups (if applicable) - data_info: Information about data processing steps - config_used: Configuration parameters actually used - processing_source: Source of input data ("processing_result", "excel_file", or "direct_data")

Enhanced PCA Plotting for Neural and Omics Data

Description

Creates publication-ready PCA plots with scientific color palettes, flexible aesthetic mapping, and multiple visualization options. Designed specifically for neural activity and omics datasets with support for complex experimental designs including treatments, genotypes, and timepoints.

Usage

pca_plots_enhanced(
  pca_output = NULL,
  plot_data = NULL,
  pca_result = NULL,
  output_dir = NULL,
  processing_result = NULL,
  experiment_name = NULL,
  grouping_variables = NULL,
  color_variable = "Treatment",
  shape_variable = "Genotype",
  secondary_shape_variable = "Timepoint",
  pannels_var = NULL,
  components = c(1, 2),
  gray_color_value = NULL,
  save_plots = FALSE,
  plot_width = 12,
  plot_height = 10,
  dpi = 300,
  verbose = TRUE
)

Arguments

Details

The function creates up to 5 different plot variants. Files are only saved when save_plots = TRUE AND output_dir is explicitly provided.

Value

A list containing:

plots

Named list of ggplot objects for each plot type

plot_data

Data.frame with plotting data and metadata

variance_explained

Numeric vector of variance explained by each component

components_plotted

Numeric vector of components used in plots

color_palette

Named character vector of colors used

shape_palette

Named numeric vector of shapes used

plotting_config

List of configuration parameters used

saved_files

Character vector of saved file paths (if save_plots = TRUE)

See Also

process_mea_flexible for MEA data processing, discover_mea_structure for automatic data structure detection

Perform MEA PCA Analysis

Description

Template function for performing PCA on MEA data

Usage

perform_mea_pca(data, variables = NULL, scale = TRUE, center = TRUE, ...)

Arguments

Value

List containing PCA results (scores, loadings, variance explained, etc.)

Perform PCA analysis (requires processed MEA data)

Plot PCA Trajectories for Time Series Data

Description

This function creates comprehensive visualizations of PCA trajectories over time, showing both individual and group-averaged trajectories with optional smoothing.

Usage

plot_pca_trajectories_general(
  pca_results,
  pc_x = "PC1",
  pc_y = "PC2",
  trajectory_grouping = NULL,
  timepoint_var = "Timepoint",
  timepoint_order = NULL,
  individual_var = "Experiment",
  point_size = 3,
  alpha = 0.7,
  line_size = 2,
  smooth_lines = FALSE,
  color_palette = NULL,
  save_plots = FALSE,
  output_dir = NULL,
  plot_prefix = "PCA_trajectories",
  width = 12,
  height = 8,
  dpi = 150,
  return_list = TRUE,
  verbose = TRUE
)

Arguments

Value

A list containing plots, trajectories, and metadata

Print Detailed PCA Variable Summary

Description

Prints formatted summary of PCA variable importance analysis

Usage

print_detailed_summary(
  top_vars,
  pc_x_top,
  pc_y_top,
  high_both,
  pc_x,
  pc_y,
  top_n,
  min_loading_threshold
)

Arguments

Value

NULL (prints to console)

Process MEA Data Flexibly

Description

This function processes Multi-Electrode Array (MEA) data files by reading CSV files, extracting measurements and metadata, applying filters, and optionally normalizing to baseline conditions. It automatically excludes standard deviation variables and handles exclusion flags to produce clean, analysis-ready datasets.

Usage

process_mea_flexible(
  main_dir,
  selected_experiments = NULL,
  selected_timepoints = NULL,
  grouping_variables = c("Treatment", "Genotype"),
  baseline_timepoint = NULL,
  unique_id_vars = c("Well", "Variable"),
  exclude_std_variables = TRUE,
  experiment_pattern = "MEA\\d+",
  timepoint_fusions = NULL,
  verbose = TRUE,
  output_path = NULL
)

Arguments

Details

The function automatically detects and excludes variables containing "Std", "std", or "STD" in their names (e.g., "Number of Spikes - Std") while keeping average/mean variables (e.g., "Number of Spikes - Avg"). Wells marked with "Ex" or "ex" in row 124 are excluded.

By default, no files are written. To save output, provide an explicit output_path parameter. Normalization creates fold-change values relative to baseline timepoint.

Process data without saving (returns data frames only) Save output by providing explicit path

Value

A list containing: - raw_data: Processed data in long format - normalized_data: Baseline-normalized data (if baseline_timepoint specified) - processing_params: List of parameters used for processing - output_path: Path to saved Excel file (only if output_path was provided) - experiment_name: Combined experiment identifier

Filter Data by Quality Metrics

Description

Filters variables and groups based on observation counts and data completeness

Usage

quality_filter(
  data,
  variable_column,
  value_column,
  grouping_columns,
  quality_threshold,
  min_observations,
  verbose
)

Arguments

Value

Filtered data frame

Examples

test_data <- data.frame(
  Variable = rep(paste0("V", 1:5), each = 20),
  Value = rnorm(100),
  Group = rep(c("A", "B"), 50)
)
filtered <- quality_filter(test_data, "Variable", "Value", "Group", 
                           0.8, 5, FALSE)

Setup Color Scheme

Description

Sets up color schemes for plotting functions

Usage

setup_color_scheme(color_scheme, custom_colors)

Arguments

Value

List of colors for plotting