advertisement

Participant Presentations Please Sign Up: • Name • Email (Onyen is fine, or …) • Are You ENRolled? • Tentative Title (???? Is OK) • When: Next Week, Early, Oct., Nov., Late Object Oriented Data Analysis Three Major Parts of OODA Applications: I. Object Definition “What are the Data Objects?” II. Exploratory Analysis “What Is Data Structure / Drivers?” III. Confirmatory Analysis / Validation Is it Really There (vs. Noise Artifact)? Yeast Cell Cycle Data, FDA View Central question: Which genes are “periodic” over 2 cell cycles? Frequency 2 Analysis Colors are Batch and Source Adjustment • For Stanford Breast Cancer Data (C. Perou) • Analysis in Benito, et al (2004) https://genome.unc.edu/pubsup/dwd/ • Adjust for Source Effects – Different sources of mRNA • Adjust for Batch Effects – Arrays fabricated at different times Source Batch Adj: PC 1-3 & DWD direction Source Batch Adj: DWD Source Adjustment NCI 60: Raw Data, Platform Colored NCI 60: Fully Adjusted Data, Platform Colored Object Oriented Data Analysis Three Major Parts of OODA Applications: I. Object Definition “What are the Data Objects?” II. Exploratory Analysis “What Is Data Structure / Drivers?” III. Confirmatory Analysis / Validation Is it Really There (vs. Noise Artifact)? Recall Drug Discovery Data 𝑛 = 262, Chemical Compounds 𝑑 = 2489, Chemical “Descriptors” Discrete Response: 0 – blue 0, 1 – red + Illustrated MargDistPlot.m (Thanks to Alex Tropsha Lab) Recall Drug Discovery Data Raw Data – PCA Scatterplot Dominated By Few Large Compounds Not Good Blue - Red Separation Recall Drug Discovery Data MargDistPlot.m – Sorted on Means Revealed Many Interesting Features Led To Data Modifcation Recall Drug Discovery Data PCA on Binary Variables Interesting Structure? Clusters? Stronger Red vs. Blue Recall Drug Discovery Data PCA on Binary Variables Deep Question: Is Red vs. Blue Separation Better? Recall Drug Discovery Data PCA on Transformed Non-Binary Variables Interesting Structure? Clusters? Stronger Red vs. Blue Recall Drug Discovery Data PCA on Transformed Non-Binary Variables Same Deep Question: Is Red vs. Blue Separation Better? Recall Drug Discovery Data Question: When Is Red vs. Blue Separation Better? Visual Approach: Train DWD to Separate Project, and View How Separated Useful View, Add Orthogonal PC Directions Recall Drug Discovery Data Raw Data – DWD & Ortho PCs Scatterplot Some Blue - Red Separation But Dominated By Few Large Compounds Recall Drug Discovery Data Binary Data – DWD & Ortho PCs Scatterplot Better Blue - Red Separation And Visualization Recall Drug Discovery Data Transform’d Non-Binary Data – DWD & OPCA Better Blue - Red Separation ??? Very Useful Visualization Caution DWD Separation Can Be Deceptive Since DWD is Really Good at Separation Important Concept: Statistical Inference is Essential Caution Toy 2-Class Example See Structure? Careful, Only PC1-4 Caution Toy 2-Class Example DWD & Ortho PCA Finds Big Separation Caution Toy 2-Class Example Actually Both Classes Are 𝑁 0, 𝐼 𝑑 = 1000 Caution Toy 2-Class Example Separation Is Natural Sampling Variation (Will Study in Detail Later) Caution Main Lesson Again: DWD Separation Can Be Deceptive Since DWD is Really Good at Separation Important Concept: Statistical Inference is Essential III. Confirmatory Analysis DiProPerm Hypothesis Test Context: 2 – sample means H0: μ+1 = μ-1 vs. H1: μ+1 ≠ μ-1 (in High Dimensions) ∃ A Large Literature. Some Highlights: o Bai & Sarandasa (2006) o Chen & Qin (2010) o Srivastava et al (2013) o Cai et al (2014) DiProPerm Hypothesis Test Context: 2 – sample means H0: μ+1 = μ-1 vs. H1: μ+1 ≠ μ-1 (in High Dimensions) Approach taken here: Wei et al (2013) Focus on Visualization via Projection (Thus Test Related to Exploration) DiProPerm Hypothesis Test Context: 2 – sample means H0: μ+1 = μ-1 vs. H1: μ+1 ≠ μ-1 Challenges: Distributional Assumptions Parameter Estimation HDLSS space is slippery DiProPerm Hypothesis Test Context: 2 – sample means H0: μ+1 = μ-1 vs. H1: μ+1 ≠ μ-1 Challenges: Distributional Assumptions Parameter Estimation Suggested Approach: Permutation test (A flavor of classical “non-parametrics”) DiProPerm Hypothesis Test Suggested Approach: Find a DIrection (separating classes) PROject the data (reduces to 1 dim) PERMute (class labels, to assess significance, with recomputed direction) DiProPerm Hypothesis Test Toy 2-Class Example Separated DWD Projections (Again 𝑁 0, 𝐼 , 𝑑 = 1000) DiProPerm Hypothesis Test Toy 2-Class Example Separated DWD Projections Measure Separation of Classes Using: Mean Difference = 6.209 DiProPerm Hypothesis Test Toy 2-Class Example Separated DWD Projections Measure Separation of Classes Using: Mean Difference = 6.209 Record as Vertical Line DiProPerm Hypothesis Test Toy 2-Class Example Separated DWD Projections Measure Separation of Classes Using: Mean Difference = 6.209 Statistically Significant??? DiProPerm Hypothesis Test Toy 2-Class Example Permuted Class Labels DiProPerm Hypothesis Test Toy 2-Class Example Permuted Class Labels Recompute DWD & Projections DiProPerm Hypothesis Test Toy 2-Class Example Measure Class Separation Using Mean Difference = 6.26 DiProPerm Hypothesis Test Toy 2-Class Example Measure Class Separation Using Mean Difference = 6.26 Record as Dot DiProPerm Hypothesis Test Toy 2-Class Example Generate 2nd Permutation DiProPerm Hypothesis Test Toy 2-Class Example Measure Class Separation Using Mean Difference = 6.15 DiProPerm Hypothesis Test Toy 2-Class Example Record as Second Dot DiProPerm Hypothesis Test . . . Repeat This 1,000 Times To Generate Null Distribution DiProPerm Hypothesis Test Toy 2-Class Example Generate Null Distribution DiProPerm Hypothesis Test Toy 2-Class Example Generate Null Distribution Compare With Original Value DiProPerm Hypothesis Test Toy 2-Class Example Generate Null Distribution Compare With Original Value Take Proportion Larger as P-Value DiProPerm Hypothesis Test Toy 2-Class Example Generate Null Distribution Compare With Original Value Not Significant DiProPerm Hypothesis Test 𝐽 = vector of 1s Another Example 𝑁1000 +0.05 ∗ 𝐽, 𝐼 𝑁1000 −0.05 ∗ 𝐽, 𝐼 PCA View DiProPerm Hypothesis Test Another Example 𝑁1000 +0.05 ∗ 𝐽, 𝐼 𝑁1000 −0.05 ∗ 𝐽, 𝐼 DWD View (Similar to 𝑁 0, 𝐼 ?) DiProPerm Hypothesis Test Another Example 𝑁1000 +0.05 ∗ 𝐽, 𝐼 𝑁1000 −0.05 ∗ 𝐽, 𝐼 DiProPerm Now Quite Significant DiProPerm Hypothesis Test Stronger Example 𝑁1000 +0.2 ∗ 𝐽, 𝐼 𝑁1000 −0.2 ∗ 𝐽, 𝐼 Even PCA Shows Class Difference DiProPerm Hypothesis Test Stronger Example 𝑁1000 +0.2 ∗ 𝐽, 𝐼 𝑁1000 −0.2 ∗ 𝐽, 𝐼 DiProPerm Very Significant DiProPerm Hypothesis Test Stronger Example 𝑁1000 +0.2 ∗ 𝐽, 𝐼 𝑁1000 −0.2 ∗ 𝐽, 𝐼 >> 5.4 above DiProPerm Very Significant Z-Score Allows Comparison DiProPerm Hypothesis Test Real Data Example: Autism Caudate Shape (sub-cortical brain structure) Shape summarized by 3-d locations of 1032 corresponding points Autistic vs. Typically Developing (Thanks to Josh Cates) DiProPerm Hypothesis Test Finds Significant Difference Despite Weak Visual Impression DiProPerm Hypothesis Test Also Compare: Developmentally Delayed No Significant Difference But Stronger Visual Impression DiProPerm Hypothesis Test Two Examples Which Is “More Distinct”? Visually Better Separation? Thanks to Katie Hoadley DiProPerm Hypothesis Test Two Examples Which Is “More Distinct”? Stronger Statistical Significance! (Reason: Differing Sample Sizes) DiProPerm Hypothesis Test Value of DiProPerm: Visual Impression is Easily Misleading (Onto HDLSS Projections, Recall 𝑁 0, 𝐼 Example) Really Need to Assess Significance DiProPerm used routinely (even for variable selection) DiProPerm Hypothesis Test Choice of Direction: Distance Weighted Discrimination (DWD) Support Vector Machine (SVM) Mean Difference Maximal Data Piling Introduced Later DiProPerm Hypothesis Test Choice of 1-d Summary Statistic: 2-sample t-stat Mean difference Median difference Area Under ROC Curve Surprising Comparison Coming Later Recall Matlab Software Posted Software for OODA DiProPerm Hypothesis Test Matlab Software: DiProPermSM.m In BatchAdjust Directory Recall Drug Discovery Data Raw Data – DWD & Ortho PCs Scatterplot Some Blue - Red Separation But Dominated By Few Large Compounds Recall Drug Discovery Data Binary Data – DWD & Ortho PCs Scatterplot Better Blue - Red Separation And Visualization Recall Drug Discovery Data Transform’d Non-Binary Data – DWD & OPCA Better Blue - Red Separation ??? Very Useful Visualization Recall Drug Discovery Data DiProPerm test of Blue vs. Red Full Raw Data Z = 10.4 Reasonable Difference Recall Drug Discovery Data DiProPerm test of Blue vs. Red Delete var = 0 & -999 Variables Z = 11.6 Slightly Stronger Recall Drug Discovery Data DiProPerm test of Blue vs. Red Binary Variables Only Z = 14.6 More Than Raw Data Recall Drug Discovery Data DiProPerm test of Blue vs. Red Non-Binary – Standardized Z = 17.3 Stronger Recall Drug Discovery Data DiProPerm test of Blue vs. Red Non-Binary – Shifted Log Transform Z = 17.9 Slightly Stronger HDLSS Asymptotics Modern Mathematical Statistics: Based on asymptotic analysis HDLSS Asymptotics Modern Mathematical Statistics: Based on asymptotic analysis I.e. Uses limiting operations Almost always lim 𝑛→∞ Workhorse Method for Much Insight: Laws of Large Numbers (Consistency) Central Limit Theorems (Quantify Errors) HDLSS Asymptotics Modern Mathematical Statistics: Based on asymptotic analysis I.e. Uses limiting operations Almost always lim 𝑛→∞ Occasional misconceptions: Indicates behavior for large samples Thus only makes sense for “large” samples Models phenomenon of “increasing data” So other flavors are useless??? HDLSS Asymptotics Modern Mathematical Statistics: Based on asymptotic analysis Real Reasons: Approximation provides insights Can find simple underlying structure In complex situations Thus various flavors are fine: lim lim lim lim 𝑛→∞ 𝑑→∞ 𝑛,𝑑→∞ 𝜎→0 Even desirable! (find additional insights) HDLSS Asymptotics Personal Observations: HDLSS world is… Surprising (many times!) [Think I’ve got it, and then …] Mathematically Beautiful (?) Practically Relevant HDLSS Asymptotics: Simple Paradoxes For 𝑑 dimensional Standard Normal dist’n: 𝑍1 𝑍 = ⋮ ~𝑁𝑑 0, 𝐼𝑑 𝑍𝑑 Where are the Data? Near Peak of Density? Thanks to: psycnet.apa.org HDLSS Asymptotics: Simple Paradoxes For 𝑑 dimensional Standard Normal dist’n: 𝑍1 𝑍 = ⋮ ~𝑁𝑑 0, 𝐼𝑑 𝑍𝑑 Euclidean Distance to Origin (as 𝑑 → ∞) (measure how close to peak) HDLSS Asymptotics: Simple Paradoxes For 𝑑 dimensional Standard Normal dist’n: 𝑍1 𝑍 = ⋮ ~𝑁𝑑 0, 𝐼𝑑 𝑍𝑑 Euclidean Distance to Origin (as 𝑑 → ∞): 𝑍 = 2 𝑛 𝑍 𝑖=1 𝑖 ~ 𝑍 = 𝑑 + 𝑂𝑝 𝑑1/2 = 𝑑 + 𝑂𝑝 1 𝜒2𝑑 HDLSS Asymptotics: Simple Paradoxes As 𝑑 → ∞, 𝑍 = 𝑑 + 𝑂𝑝 1 - Data lie roughly on surface of sphere, with radius 𝑑 - Yet origin is point of highest density??? - Paradox resolved by: density w. r. t. Lebesgue Measure HDLSS Asymptotics: Simple Paradoxes - Paradox resolved by: density w. r. t. Lebesgue Measure Consider Volume of Unit Sphere in ℝ𝑑 Find As: Integral In Sph’l Coordinates 𝑉𝑑 = 𝐽𝑑𝜃1 ⋯ 𝑑𝜃𝑑 𝑑𝑟 HDLSS Asymptotics: Simple Paradoxes - Paradox resolved by: density w. r. t. Lebesgue Measure Consider Volume of Unit Sphere in ℝ𝑑 Find As: Integral In Sph’l Coordinates 𝑉𝑑 = 𝐽𝑑𝜃1 ⋯ 𝑑𝜃𝑑 𝑑𝑟 Look At Integrand w.r.t. 𝑟 Can Show: Puts ~ All Weight Near 𝑟 = 1 HDLSS Asymptotics: Simple Paradoxes - Paradox resolved by: density w. r. t. Lebesgue Measure Lebesgue Measure Pushes Mass Out Density Pulls Data In 𝑑 − 1 2 Is The Balance Point HDLSS Asymptotics: Simple Paradoxes As 𝑑 → ∞, 𝑍 = 𝑑 + 𝑂𝑝 1 Important Philosophical Consequence: ∄ “Average People” Parents Lament: Why Can’t I Have Average Children? Theorem: Impossible (over many factors)! HDLSS Asymptotics: Simple Paradoxes Distance tends to non-random constant: 𝑍1 − 𝑍2 = 2𝑑 + 𝑂𝑝 1 Factor 2, since 𝑠𝑑 𝑋1 − 𝑋2 = 𝑠𝑑 𝑋1 2 + 𝑠𝑑 𝑋2 Can extend to 𝑍1 , ⋯ 𝑍𝑛 All points are equidistant (We can only perceive 3 dimensions) 2 HDLSS Asymptotics: Simple Paradoxes Ever Wonder Why? o Perceptual System from Ancestors o They Needed to Find Food o Food Exists in 3-d World (We can only perceive 3 dimensions) HDLSS Asymptotics: Simple Paradoxes For 𝑑 dim’al Standard Normal dist’n: 𝑍1 indep. of 𝑍2 ~𝑁𝑑 0, 𝐼𝑑 High dim’al Angles (as 𝑑 → ∞): As vectors from the Origin cos −1 𝑢1 𝑡 𝑢2 Thanks to: members.tripod.com HDLSS Asymptotics: Simple Paradoxes For 𝑑 dim’al Standard Normal dist’n: 𝑍1 indep. of 𝑍2 ~𝑁𝑑 0, 𝐼𝑑 High dim’al Angles (as 𝑑 → ∞): 𝐴𝑛𝑔𝑙𝑒 𝑍1 , 𝑍2 = 90° + 𝑂𝑝 𝑑 −1/2 - Everything is orthogonal??? HDLSS Asy’s: Geometrical Represent’n Assume 𝑍1 , ⋯ 𝑍1 ~𝑁𝑑 0, 𝐼𝑑 , let 𝑑 → ∞ Study Subspace Generated by Data Hall, Marron & Neeman (2005) HDLSS Asy’s: Geometrical Represent’n Assume 𝑍1 , ⋯ 𝑍1 ~𝑁𝑑 0, 𝐼𝑑 , let 𝑑 → ∞ Study Subspace Generated by Data Hyperplane through 0, of dimension 𝑛 Hall, Marron & Neeman (2005) HDLSS Asy’s: Geometrical Represent’n Assume 𝑍1 , ⋯ 𝑍1 ~𝑁𝑑 0, 𝐼𝑑 , let 𝑑 → ∞ Study Subspace Generated by Data Hyperplane through 0, of dimension 𝑛 Points are “nearly equidistant to 0”, & dist 𝑑 Hall, Marron & Neeman (2005) HDLSS Asy’s: Geometrical Represent’n Assume 𝑍1 , ⋯ 𝑍1 ~𝑁𝑑 0, 𝐼𝑑 , let 𝑑 → ∞ Study Subspace Generated by Data Hyperplane through 0, of dimension 𝑛 Points are “nearly equidistant to 0”, & dist 𝑑 Within plane, can “rotate towards 𝑑 × Unit Simplex” Hall, Marron & Neeman (2005) HDLSS Asy’s: Geometrical Represent’n Assume 𝑍1 , ⋯ 𝑍1 ~𝑁𝑑 0, 𝐼𝑑 , let 𝑑 → ∞ Study Subspace Generated by Data Hyperplane through 0, of dimension 𝑛 Points are “nearly equidistant to 0”, & dist 𝑑 Within plane, can “rotate towards 𝑑 × Unit Simplex” All Gaussian data sets are: “near Unit Simplex Vertices”!!! (modulo rotation) Hall, Marron & Neeman (2005) HDLSS Asy’s: Geometrical Represent’n Assume 𝑍1 , ⋯ 𝑍1 ~𝑁𝑑 0, 𝐼𝑑 , let 𝑑 → ∞ Study Subspace Generated by Data Hyperplane through 0, of dimension 𝑛 Points are “nearly equidistant to 0”, & dist 𝑑 Within plane, can “rotate towards 𝑑 × Unit Simplex” All Gaussian data sets are: “near Unit Simplex Vertices”!!! “Randomness” appears only in rotation of simplex Hall, Marron & Neeman (2005) HDLSS Asy’s: Geometrical Represent’n Assume 𝑍1 , ⋯ 𝑍1 ~𝑁𝑑 0, 𝐼𝑑 , let 𝑑 → ∞ Study Hyperplane Generated by Data 𝑛 − 1 dimensional hyperplane Points are pairwise equidistant, dist ~ 2𝑑 Points lie at vertices of: 2𝑑 × “regular 𝑛 − hedron” Again “randomness in data” is only in rotation Surprisingly rigid structure in random data? HDLSS Asy’s: Geometrical Represen’tion Simulation View: study “rigidity after rotation” • Simple 3 point data sets • In dimensions d = 2, 20, 200, 20000 • Generate hyperplane of dimension 2 • Rotate that to plane of screen • Rotate within plane, to make “comparable” • Repeat 10 times, use different colors HDLSS Asy’s: Geometrical Represen’tion Simulation View: Shows “Rigidity after Rotation” HDLSS Asy’s: Geometrical Represen’tion Straightforward Generalizations: non-Gaussian data: non-independent: only need moments? use “mixing conditions” Mild Eigenvalue condition on Theoretical Cov. (Ahn, Marron, Muller & Chi, 2007) ⋮ All based on simple “Laws of Large Numbers”