Executive Summary
Dimensionality and dominant structure across 150 observations
The customer dataset has 2 real dimensions when compressed to 80% variance, though 3 dimensions are suggested by the Kaiser criterion (eigenvalue > 1). PC1 alone captures 48.9% of total variation, driven by Ad Spend, Store Visits, and Basket Size. Adding PC2 (32.2% variance) reaches 81.2% cumulatively—the effective dimensionality threshold. The remaining 4 components contribute only 18.8% and mostly encode noise. This high compressibility means most customer variation is captured by two independent patterns: a spending/engagement axis and a service-quality axis.
Analysis Overview
Principal component analysis of 6 standardized features across 150 observations.
This analysis applies PCA to 6 standardized customer features across 150 observations to identify the dataset's true dimensionality. Standardization (mean 0, variance 1) ensures that features measured on different scales—like ad spend in dollars versus return rate as a percentage—don't artificially dominate the decomposition. Loadings reveal which original features drive each principal component: features with similar or opposing loadings on the same component vary together, forming the underlying structure. All 150 rows and 6 features were usable after median imputation of missing values.
Data Quality
Feature typing, imputation, and exclusions before PCA.
All 150 observations were retained; no rows were removed during quality checks. Missing values were imputed using each column's median, a conservative approach that preserves the marginal distribution of each feature. Standardization was then applied, transforming each feature to mean 0 and variance 1. This step is critical: it prevents high-variance features (e.g., ad spend in absolute dollars) from dominating the components simply because they have larger raw numbers. All 6 mapped feature columns—Ad Spend, Store Visits, Basket Size, Support Tickets, Return Rate, Noise Metric—were numeric and usable; no columns were excluded as constant or empty. The resulting standardized matrix is the input to PCA.
Variance Explained by Component
How much of the total variation each principal component captures (scree chart).
The scree chart reveals a clear two-stage drop-off. PC1 carries 48.9% of variance and PC2 adds 32.2%, together reaching 81.2%—the practical threshold for data compression. PC3 contributes 16.7%, bringing the cumulative to 97.8%, but this third component's marginal gain is modest. Beyond PC3, the curve flattens dramatically: PC4, PC5, and PC6 contribute only 0.8%, 0.7%, and 0.6% respectively, totaling 1.9%. This tail confirms that dimensionality beyond 2–3 is noise. The elbow is sharp between PC2 and PC3, supporting the conclusion that 2 components capture the essential signal.
Component Summary
Eigenvalues and variance shares for the leading components.
| Component | Eigenvalue | Variance PCT | Cumulative PCT |
|---|---|---|---|
| PC1 | 2.936 | 48.9 | 48.9 |
| PC2 | 1.933 | 32.2 | 81.2 |
| PC3 | 1 | 16.7 | 97.8 |
| PC4 | 0.051 | 0.8 | 98.7 |
| PC5 | 0.04 | 0.7 | 99.4 |
| PC6 | 0.039 | 0.6 | 100 |
PC1 has an eigenvalue of 2.936 (48.9% variance), PC2 has 1.933 (32.2%), and PC3 has exactly 1.0 (16.7%). These three components pass the Kaiser criterion (eigenvalue ≥ 1), meaning each carries at least as much information as a single standardized original feature. PC4's eigenvalue drops to 0.051, a 50-fold decrease, signaling the transition to noise. For practical modeling, 2 components reach 81.2% cumulative variance (a common 80% threshold), while 3 reach 97.8%. The choice depends on tolerance for information loss: 2 components compress the data aggressively; 3 retain nearly all signal. Beyond 3, eigenvalues are negligible.
Feature Contributions
Which features define the first two components, scaled 0-100.
Basket Size (100) and Store Visits (98.5) dominate the main structure, followed closely by Ad Spend (95.3). These three features define the dataset's primary axis of variation. Support Tickets (89.5) and Return Rate (88.9) contribute substantially to the secondary structure (PC2). Noise Metric ranks last at 5.2, indicating it varies largely independently of the dominant customer patterns captured by PC1 and PC2. This ranking reflects each feature's combined loading strength on the first two components, weighted by their variance shares. Features near 100 are tightly coupled to the main customer segmentation; those near 5 are essentially orthogonal to it.
Observation Map (PC1 vs PC2)
Every observation positioned on the two dominant components.
Observations are scattered across PC1 (horizontal, 48.9% variance) and PC2 (vertical, 32.2% variance), together spanning 81.2% of total variation. Segment labels (Budget, Mid, Premium) show visible separation: Premium customers cluster toward positive PC1 (higher ad spend, store visits, basket size), while Budget customers cluster toward negative PC1. PC2 separates primarily on Return Rate: observations with negative PC2 scores tend to have lower return rates, while positive PC2 indicates higher return rates and support tickets. The separation is clear but not perfect—some overlap occurs, indicating that Segment differences also exist on the third and higher components not shown here.
Component Loadings
How strongly each feature defines each leading component.
| Feature | Pc1 | Pc2 | Pc3 |
|---|---|---|---|
| Ad Spend | 0.572 | 0.067 | -0.037 |
| Store Visits | 0.57 | 0.1 | -0.01 |
| Basket Size | 0.566 | 0.121 | 0.025 |
| Support Tickets | 0.12 | -0.695 | 0.032 |
| Return Rate | 0.115 | -0.697 | 0.028 |
| Noise Metric | -0.006 | -0.042 | -0.998 |
PC1 is defined by three features with nearly identical loadings: Ad Spend (0.572), Store Visits (0.57), and Basket Size (0.566). These move together, forming a single "engagement and value" axis. PC2 is driven by Support Tickets (−0.695) and Return Rate (−0.697), which load together negatively, forming a "service quality" axis. PC3 is almost entirely defined by Noise Metric (−0.998), capturing idiosyncratic noise. Features with near-zero loadings on leading components (e.g., Noise Metric on PC1 and PC2) carry independent information. The tight alignment of Ad Spend, Store Visits, and Basket Size on PC1 suggests these customer metrics are tightly correlated in the raw data.
Methodology
Statistical methodology and diagnostics for Principal Component Analysis (PCA)
Statistical Method
Standard-library analysis: principal component analysis on the numeric columns you choose. Shows how many independent dimensions your data really has, which features move together, and how observations spread across the dominant components — with a scree chart, component summary, loadings, feature contributions, and an observation map with optional group coloring. Works on any dataset: map 2 or more numeric features.
- Features are numeric (or cleanly convertible) and linearly related
- Standardization is appropriate (each feature weighted equally)
- Components are linear combinations — non-linear structure needs t-SNE/UMAP
- Components require domain interpretation; PCA does not name them
- Categorical features are excluded (only an optional label for coloring)
Analysis Code
Complete R source code for this analysis
Principal Component Analysis — Structure Finder
Runs PCA on the numeric feature columns the user selects, revealing how many independent dimensions the data really has, which features move together, and how observations spread across the leading components.
Why This Method?
PCA is the standard first tool for understanding high-dimensional data: it compresses correlated features into a few independent components, quantifies how much signal each carries, and exposes the groups of features that vary together — before any modelling is attempted.
What This Analysis Covers
- Variance explained per component (scree) + how many components you need
- Component summary table (eigenvalues, cumulative variance)
- Which features drive the leading components (loadings + contributions)
- Observation map on the first two components (optional group coloring)
Standard Library
Platform standard-library module (LAT-1441): runs on ANY dataset via the semantic mapping {feature_1..feature_N, optional label}. All narrative is derived from the user's own column names and computed values.
suppressPackageStartupMessages(library(htmltools))
suppressPackageStartupMessages(library(jsonlite))
suppressPackageStartupMessages(library(plotly))
suppressPackageStartupMessages(library(DT))
suppressPackageStartupMessages(library(htmlwidgets))
suppressPackageStartupMessages(library(arrow))
suppressPackageStartupMessages(library(knitr))
suppressPackageStartupMessages(library(rmarkdown))
suppressPackageStartupMessages(library(dplyr))
suppressPackageStartupMessages(library(tidyr))
suppressPackageStartupMessages(library(ggplot2))
suppressPackageStartupMessages(library(stringr))
suppressPackageStartupMessages(library(lubridate))
suppressPackageStartupMessages(library(broom))
suppressPackageStartupMessages(library(Matrix))
suppressPackageStartupMessages(library(cluster))
suppressPackageStartupMessages(library(data.table))Step 1: Discover mapped feature columns
initial_rows <- nrow(df)
feat_cols <- grep("^feature_[0-9]+$", names(df), value = TRUE)
feat_cols <- feat_cols[order(as.integer(sub("^feature_", "", feat_cols)))]
if (length(feat_cols) < 2) {
stop("column_mapping must map at least two feature columns(feature_1, feature_2)")
}
feature_names <- setNames(humanize_semantic(feat_cols, col_map), feat_cols)
has_label <- "label" %in% names(df)
label_name <- if (has_label) humanize_semantic("label", col_map) else ""Step 2: Coerce features numeric (95% rule); impute median; drop unusable
dropped_features <- character(0)
for (fc in feat_cols) {
v <- df[[fc]]
if (!is.numeric(v)) {
conv <- suppressWarnings(as.numeric(as.character(v)))
n_orig <- sum(!is.na(v) & as.character(v) != "")
if (n_orig > 0 && sum(!is.na(conv)) >= 0.95 * n_orig) {
df[[fc]] <- conv
} else {
dropped_features <- c(dropped_features, fc)
next
}
}
v <- df[[fc]]
med <- median(v, na.rm = TRUE)
if (is.na(med)) { dropped_features <- c(dropped_features, fc); next }
v[is.na(v)] <- med
df[[fc]] <- v
if (isTRUE(var(v) == 0) || is.na(var(v))) {
dropped_features <- c(dropped_features, fc)
}
}
used_features <- setdiff(feat_cols, dropped_features)
if (length(used_features) < 2) {
stop(paste0(
"PCA needs at least two usable numeric feature columns; only ",
length(used_features), " remained after cleaning. Excluded: ",
paste(feature_names[dropped_features], collapse = ", ")
))
}
X <- df[, used_features, drop = FALSE]
final_rows <- nrow(X)
rows_removed <- initial_rows - final_rows
if (final_rows < 10) {
stop(sprintf("Only %d usable rows — PCA needs at least 10.", final_rows))
}Step 3: PCA on standardized features
pca <- prcomp(X, center = TRUE, scale. = TRUE)
eig <- pca$sdev^2
var_pct <- 100 * eig / sum(eig)
cum_pct <- cumsum(var_pct)
n_comp_all <- length(eig)
n_show <- min(n_comp_all, 10)
variance_df <- data.frame(
component = paste0("PC", seq_len(n_show)),
variance_pct = round(var_pct[seq_len(n_show)], 1),
cumulative_pct = round(cum_pct[seq_len(n_show)], 1),
stringsAsFactors = FALSE
)
components_df <- data.frame(
component = paste0("PC", seq_len(n_show)),
eigenvalue = round(eig[seq_len(n_show)], 3),
variance_pct = round(var_pct[seq_len(n_show)], 1),
cumulative_pct = round(cum_pct[seq_len(n_show)], 1),
stringsAsFactors = FALSE
)
n_for_80 <- which(cum_pct >= 80)[1]
n_for_90 <- which(cum_pct >= 90)[1]
n_kaiser <- sum(eig > 1)Step 4: Loadings (signed, first 3 PCs) with humanized feature names
n_pcs_load <- min(3, n_comp_all)
L <- pca$rotation[, seq_len(n_pcs_load), drop = FALSE]
loadings_df <- data.frame(
feature = unname(feature_names[used_features]),
stringsAsFactors = FALSE
)
for (j in seq_len(n_pcs_load)) {
loadings_df[[paste0("pc", j)]] <- round(L[, j], 3)
}Step 5: Feature contributions — |loading| weighted by PC1+PC2 variance, 0-100
w <- var_pct[seq_len(min(2, n_comp_all))]
contrib_raw <- as.vector(abs(L[, 1]) * w[1] +
(if (n_comp_all >= 2) abs(L[, 2]) * w[2] else 0))
contributions_df <- data.frame(
feature = unname(feature_names[used_features]),
contribution = round(100 * contrib_raw / max(contrib_raw), 1),
stringsAsFactors = FALSE
)
contributions_df <- contributions_df[order(-contributions_df$contribution), , drop = FALSE]
rownames(contributions_df) <- NULLStep 6: Scores map (PC1 vs PC2), <=1000 sample; label always present
set.seed(42)
sidx <- if (final_rows > 1000) sample(final_rows, 1000) else seq_len(final_rows)
lab_vals <- if (has_label) {
lv <- as.character(df$label)
lv[is.na(lv) | trimws(lv) == ""] <- "Missing"
tab <- sort(table(lv), decreasing = TRUE)
if (length(tab) > 10) {
keep <- names(tab)[1:10]
lv[!(lv %in% keep)] <- "Other"
}
lv
} else {
rep("All rows", final_rows)
}
scores_df <- data.frame(
pc1 = round(pca$x[sidx, 1], 3),
pc2 = if (n_comp_all >= 2) round(pca$x[sidx, 2], 3) else 0,
label = lab_vals[sidx],
stringsAsFactors = FALSE
)
rownames(scores_df) <- NULLStep 7: Narrative anchors
pc1_order <- order(-abs(L[, 1]))
top_pc1_features <- unname(feature_names[used_features])[pc1_order][
seq_len(min(3, length(used_features)))]
pc2_order <- if (n_comp_all >= 2) order(-abs(L[, 2])) else pc1_order
top_pc2_feature <- unname(feature_names[used_features])[pc2_order][1]
metrics <- list(
`Observations` = final_rows,
`Features Analysed` = length(used_features),
`PC1 Variance %` = round(var_pct[1], 1),
`Components for 80%` = as.integer(n_for_80),
`Kaiser Components` = as.integer(n_kaiser),
`Top PC1 Feature` = top_pc1_features[1]
)
json_output <- list(
answer = paste0(
"PCA on ", length(used_features), " features across ",
format(final_rows, big.mark = ","), " rows: PC1 explains ",
round(var_pct[1], 1), "% of variance(",
n_for_80, " component(s) reach 80%, Kaiser criterion keeps ",
n_kaiser, "). PC1 is driven by ",
paste(top_pc1_features, collapse = ", "), "."
),
cards = lapply(
c("tldr", "overview", "preprocessing", "variance_explained",
"components_table", "feature_contributions", "scores_scatter",
"loadings_table"),
function(cid) list(id = cid, metrics = metrics)
)
)
list(
initial_rows = initial_rows,
final_rows = final_rows,
rows_removed = rows_removed,
feature_names = feature_names,
used_features = used_features,
dropped_features = dropped_features,
has_label = has_label,
label_name = label_name,
pca = pca,
variance_df = variance_df,
components_df = components_df,
loadings_df = loadings_df,
contributions_df = contributions_df,
scores_df = scores_df,
n_for_80 = n_for_80,
n_for_90 = n_for_90,
n_kaiser = n_kaiser,
top_pc1_features = top_pc1_features,
top_pc2_feature = top_pc2_feature,
var_pct = var_pct,
metrics = metrics,
json_output = json_output
)
}Compute shared resources
shared <- compute_shared(df, params, col_map)