Dissertation Index
Author: Vurkaç, Mehmet Title: Prestructuring Multilayer Perceptrons based on Information-Theoretic Modeling of a Partido-Alto-based Grammar for Afro-Brazilian Music: Enhanced Generalization and Principles of Parsimony, including an Investigation of Statistical Paradigms Institution: Portland State University Begun: June 2008 Completed: December 2011 Abstract: The present study shows that prestructuring based on domain knowledge leads to statistically significant generalization-performance improvement in artificial neural networks (NNs) of the multilayer perceptron (MLP) type, specifically in the case of a noisy real-world problem with numerous interacting variables. The prestructuring of MLPs based on knowledge of the structure of a problem domain has previously been shown to improve generalization performance. However, the problem domains for those demonstrations suffered from significant shortcomings: 1) They were purely logical problems, and 2) they contained small numbers of variables in comparison to most data-mining applications today. Two implications of the former were a) the underlying structure of the problem was completely known to the network designer by virtue of having been conceived for the problem at hand, and b) noise was not a significant concern in contrast with real-world conditions. As for the size of the problem, neither computational resources nor mathematical modeling techniques were advanced enough to handle complex relationships among more than a few variables until recently, so such problems were left out of the mainstream of prestructuring investigations. In the present work, domain knowledge is built into the solution through Reconstructability Analysis, a form of information-theoretic modeling, which is used to identify mathematical models that can be transformed into a graphic representation of the problem domain\'s underlying structure. Employing the latter as a pattern allows the researcher to prestructure the MLP, for instance, by disallowing certain connections in the network. Prestructuring reduces the set of all possible maps (SAPM) that are realizable by the NN. The reduced SAPM--according to the Lendaris-Stanley conjecture, conditional probability, and Occam\'s razor--enables better generalization performance than with a fully connected MLP that has learned the same I/O mapping to the same extent. In addition to showing statistically significant improvement over the generalization performance of fully connected networks, the prestructured networks in the present study also compared favorably to both the performance of qualified human agents and the generalization rates in classification through Reconstructability Analysis alone, which serves as the alternative algorithm for comparison. Keywords: Clave, machine learning, prestructuring, samba, computational intelligence, meter, rhythm, offbeatness, syncopation, neural networks, information theory, computer science, perceptrons, machine listening TOC: CHAPTER I. INTRODUCTION 1.1 Problem Statement 1.2 Motivation and Purpose 1.3 Objective of the Study 1.4 Research Conducted 1.5 Contributions 1.6 Significance and Rationale 1.7 Assumptions and Hypotheses 1.8 Scope, Limitations, and Constraints 1.9 Teacher Models 1.10 Selection of Holdout Data 1.11 Output Encoding 1.12 Number of Hidden Layers 1.13 Number of Hidden-Layer Elements 1.14 Criteria for Model Evaluation and Selection in RA 1.15 Output Thresholding in MLPs 1.16 RA-Search Heuristics 1.17 Definitions 1.17.1 Multilayer Perceptron (MLP) 1.17.2 Generalization 1.17.3 Clave Direction 1.17.4 Tactus and Tatum 1.17.5 Syncopation and Offbeatness 1.17.6 Attack-Point Rhythm CHAPTER II. BACKGROUND 2.1 Science and Computational Intelligence 2.2 An Overview of Machine Learning and Statistical Learning 2.2.1 Types of Cross-Validation 2.2.2 Holdout Cross-Validation: The Basics 2.2.3 Holdout Cross-Validation for Design 2.2.4 k-Fold Cross-Validation: The Basics 2.2.5 k-Fold Cross-Validation for Design 2.2.6 Statistical Soundness 2.2.7 Holdout Cross-Validation for Assessment 2.2.8 k-Fold Cross-Validation for Assessment 2.2.9 Motivation for Cross-Validation 2.2.10 The Overall Holdout 2.2.11 Performance Metric 2.3 Artificial Neural Networks 2.3.1 Historical Background on Neural Nets 2.3.2 The General Algorithm for the Delta Rule 2.3.3 A Brief Survey of Other Neural-Net Types 2.4 Reconstructability Analysis 2.4.1 RA and Occam3 2.4.2 Historical Reference 2.4.3 RA Formalism 2.4.4 Aspects of Statistics Relevant to RA 2.5 Clave and Clave Direction, with a note on labels 2.6 Information Theory, Reconstructability Analysis, Multilayer Perceptrons, and the Clave Concept: The Connection 2.7 Researcher’s Background and Qualifications 2.7.1 Academic Background as Student 2.7.2 Academic Background as Instructor CHAPTER III. LITERATURE REVIEW 3.1 Neural-Network Prestructuring 3.2 Literature Review for Neural-Network Prestructuring 3.3 Current Practice in Neural-Network Optimization CHAPTER IV: METHODOLOGY 4.1 Introduction 4.2 Notation and Data 4.2.1 Occam3 Notation 4.2.2 Music Notation 4.3 Nature of the Data 4.4 The Determination of Data Splits 4.5 Experiment-Design Rationale 4.5.1 Factorial Designs 4.5.2 Statistical Power, Significance and Hypothesis Testing 4.5.3 Control Design 4.5.4 Assumptions, Justified and Unjustified 4.6 Experimental Process 4.6.1 The General Experimental Process 4.6.2 The Processes of Data Acquisition and Preparation 4.6.3 Data Acquisition: Classification 4.6.4 Training-and-Test Regimes 4.6.5 RA Search, Fitting and Selection CHAPTER V. APPLICATION AREAS 5.1 Neural-Network Prestructuring 5.2 Music Training 5.2.1 The Growing Relevance of Clave 5.2.2 A Note on Terminology 5.2.3 Introduction to Clave 5.2.4 Systems of Musical Harmony 5.2.5 Clave as a Principle of Rhythmic Harmony 5.2.6 A Closer Look at Some Intricacies of Clave 5.2.7 Why is the Study of Clave Important? 5.3. Clave and Music Technology 5.3.1 Why a High-Tech Solution to Clave Training? 5.3.2 Current Research and Appropriate Technologies for Machine Recognition of Clave Direction 5.3.3 A high-level framework for realizing automated recognition of clave direction 5.4. Clave Analysis as a Music-Production Feature 5.4.1 Music-Making 5.4.2 Music Training 5.4.3 Music Teaching 5.5 Conclusion 5.6 JMTE-Suggested Citation for the Published Article CHAPTER VI: FINDINGS/RESULTS 6.1 Results of Neural-Net Design 6.2 Tables of Numerical Results 6.3 Preliminary Statistical Analysis 6.4 Full Statistical Analysis 6.5 Discussion of Findings CHAPTER VII: CONCLUSION 7.1 Prestructuring Performance 7.2 Statistical Significance 7.3 Implications for the Use of Reconstructability Analysis in the NN Field 7.4 General Implications for Neural Networks 7.5 Musical and Music-Technological Insight 7.6 Recommendations Concerning the Methodology of Future Studies and Focus of Future Development 7.7 Limitations 7.8 Possible Sources of Error 7.9 Shortcomings of the Dissertation 7.10 Strengths of the Dissertation 7.11 Summary APPENDIX A: A Cross-Cultural Grammar for Temporal Harmony in Afro-Latin Musics: Clave, Partido-Alto, and Other Timelines A.1 Introduction A.2 The Basic Claves (Clave-Proper) A.2.1 The (Afro-Cuban) Son Clave: A seemingly obvious starting point A.2.2 The So-Called Brazilian (Bossa) Clave A.2.3 A Graphical Analogy for Clave Direction A.2.4 The Clave Prototypes: Fundamental Rhythms for Clave Direction A.2.5 Wide-Sense Clave Consciousness A.2.6 Relative Nature of Clave Direction in Other Afro-American Rhythms A.2.7 Conclusion APPENDIX B: Information Theory B.1 Uncertainty and Information B.2 Entropy B.3 Transmission, Mutual Information, Information Distance and the KullbackLeibler Divergence APPENDIX C: Literature Review of Music Information Retrieval and Digital (Audio) Signal Processing C.1 The Reviews C.2 Conclusion APPENDIX D: Statistics, Social Responsibility, and the Enhanced Scientific Method D.1 Why Statistics? D.2 The Role of Statistics in the Scientific Method: Statistics as Meta-Science D.2.1 Misuse of Statistics in Science, Medicine and Technology D.2.2 Misuse of Statistical Techniques in Model Evaluation D.2.3 What Can Be Done? D.3 Statistics, Critical Thinking, Science, and Social Responsibility D.4 The Scientific Method, Past and Present D.4.1 Accuracy D.4.2 Objectivity D.4.3 Skepticism D.4.4 Open-mindedness D.4.5 Additional Concerns, Techniques, and Principles APPENDIX E: The Data-Acquisition Process E.1 Data Acquisition: Golden Ears E.2 Data Acquisition: Classification E.3 Data Acquisition: Golden Ears E.4 Bootstrap Process: Misclassification and NN Feedback E.5 Bootstrap Process: Context-Dependent Patterns E.6 Final Approach APPENDIX F: Brazilian Glossary, Clave Terminology, Musical Definitions, and Samba Concepts F.1 Author’s Original Samba Glossary F.2 Clave Terminology throughout the Diaspora F. 3 Musicological Definitions, Brazilian, European and Global F.3.1 Definitions of Music F.3.2 The Elements of Music F.3.3 Time, Rhythm and Meter F.3.4 Brazilian Musical Definitions from Jorge Alabê APPENDIX G: Northern/Non-Northern vs. Western/Non-Western APPENDIX H: Occurrences of Clave-Type Patterns in Music around the World, and Recommended Listening H.1 Examples of Clave Direction in Brazilian Musics H.2 Accessible Examples of Clave in other African-based Music, and Jazz H.3 Clave-type Timelines or Patterns found in Non-Clave Musics H.4 Recommended Listening in African and Afro-Latin Music APPENDIX I: Author’s Musical Background and Qualifications APPENDIX J: A Survey of Notational Conventions and Preferences for Meter and Time Signature for Musics of the African Diaspora J.1 Purpose J.2 Data J.3 Findings J.4 Conclusion APPENDIX K: The automate.dat Script for Neural-Net Experiments in NeuralWare’s NeuralWorks APPENDIX L: The AutomateNeuralWorks.vbs Script for Exercising Network Configurations with Multiple Seeds APPENDIX M: C Code for the Top-Generalizing BIC-based Network APPENDIX N: Sample Data in the Firm-Teacher Context APPENDIX O: Random Seeds Contact: mehmet.vurkac@gmail.com |