ARTIFICIAL INTELLIGENCE AND ROBOTICS
CONTENTS INTRODUCTION TO ARTI?CIAL INTELLIGENCE HISTORY OF ARTI?CIAL INTELLIGENCE WHAT IS ARTI?CIAL INTELLIGENCE? EMERGENCE OF AI COGNITIVE SCIENCE AND AI LOGICAL APPROACH TO AI KNOWLEDGE-BASED SYSTEMS PROBABILISTIC APPROACH TO AI EVOLUTIONARY COMPUTATION AND SWARM INTELLIGENCE NEURAL NETWORKS & DEEP LEARNING A RETURN TO CREATING HAL OUTLINE OF THIS BOOK I LOGICAL INTELLIGENCE PROPOSITIONAL LOGIC BASICS OF PROPOSITIONAL LOGIC SYNTAX SEMANTICS TAUTOLOGIES AND LOGICAL IMPLICATION LOGICAL ARGUMENTS DERIVATION SYSTEMS RESOLUTION NORMAL FORMS DERIVATIONS USING RESOLUTION RESOLUTION ALGORITHM ARTI?CIAL INTELLIGENCE APPLICATIONS WUMPUS WORLD DISCUSSION AND FURTHER READING FIRST-ORDER LOGIC BASICS OF FIRST-ORDER LOGIC VALIDITY AND LOGICAL IMPLICATION MODUS PONENS FOR FIRST-ORDER LOGIC WUMPUS WORLD REVISITED PLANNING CERTAIN KNOWLEDGE REPRESENTATION TAXONOMIC KNOWLEDGE SEMANTIC NETS MODEL OF HUMAN ORGANIZATION OF KNOWLEDGE FRAMES FRAME DATA STRUCTURE PLANNING A TRIP USING FRAMES NONMONOTONIC LOGIC CIRCUMSCRIPTION DEFAULT LOGIC DI?CULTIES LEARNING DETERMINISTIC MODELS SUPERVISED LEARNING REGRESSION SIMPLE LINEAR REGRESSION MULTIPLE LINEAR REGRESSION OVER?TTING AND CROSS VALIDATION PARAMETER ESTIMATION ESTIMATING THE PARAMETERS FOR SIMPLE LINEAR REGRESSION GRADIENT DESCENT LOGISTIC REGRESSION AND GRADIENT DESCENT STOCHASTIC GRADIENT DESCENT LEARNING A DECISION TREE INFORMATION THEORY INFORMATION GAIN AND THE ID3 ALGORITHM OVER?TTING II PROBABILISTIC INTELLIGENCE PROBABILITY PROBABILITY BASICS PROBABILITY SPACES CONDITIONAL PROBABILITY AND INDEPENDENCE BAYES’ THEOREM RANDOM VARIABLES PROBABILITY DISTRIBUTIONS OF RANDOM VARIABLES INDEPENDENCE OF RANDOM VARIABLES MEANING OF PROBABILITY RELATIVE FREQUENCY APPROACH TO PROBABILITY SUBJECTIVE APPROACH TO PROBABILITY RANDOM VARIABLES IN APPLICATIONS PROBABILITY IN THE WUMPUS WORLD UNCERTAIN KNOWLEDGE REPRESENTATION INTUITIVE INTRODUCTION TO BAYESIAN NETWORKS PROPERTIES OF BAYESIAN NETWORKS DE?NITION OF A BAYESIAN NETWORK REPRESENTATION OF A BAYESIAN NETWORK CAUSAL NETWORKS AS BAYESIAN NETWORKS CAUSALITY CAUSALITY AND THE MARKOV CONDITION MARKOV CONDITION WITHOUT CAUSALITY INFERENCE IN BAYESIAN NETWORKS EXAMPLES OF INFERENCE INFERENCE ALGORITHMS AND PACKAGES INFERENCE USING NETICA NETWORKS WITH CONTINUOUS VARIABLES GAUSSIAN BAYESIAN NETWORKS HYBRID NETWORKS OBTAINING THE PROBABILITIES DI?CULTY INHERENT IN MULTIPLE PARENTS BASIC NOISY OR-GATE MODEL LEAKY NOISY OR-GATE MODEL FURTHER MODELS LARGE-SCALE APPLICATION: PROMEDAS ADVANCED PROPERTIES OF BAYESIAN NETWORKS ENTAILED CONDITIONAL INDEPENDENCIES EXAMPLES OF ENTAILED CONDITIONAL INDEPENDENCIES D-SEPARATION FAITHFULNESS UNFAITHFUL PROBABILITY DISTRIBUTIONS FAITHFULNESS CONDITION MARKOV EQUIVALENCE MARKOV BLANKETS AND BOUNDARIES DECISION ANALYSIS DECISION TREES SIMPLE EXAMPLES SOLVING MORE COMPLEX DECISION TREES IN?UENCE DIAGRAMS REPRESENTING DECISION PROBLEMS WITH IN?UENCE DIAGRAMS SOLVING IN?UENCE DIAGRAMS TECHNIQUES FOR SOLVING IN?UENCE DIAGRAMS SOLVING IN?UENCE DIAGRAMS USING NETICA MODELING RISK PREFERENCES EXPONENTIAL UTILITY FUNCTION ASSESSING R ANALYZING RISK DIRECTLY USING THE VARIANCE TO MEASURE RISK RISK PRO?LES DOMINANCE GOOD DECISION VERSUS GOOD OUTCOME SENSITIVITY ANALYSIS VALUE OF INFORMATION EXPECTED VALUE OF PERFECT INFORMATION EXPECTED VALUE OF IMPERFECT INFORMATION ACADEMICS BUSINESS AND FINANCE CAPITAL EQUIPMENT COMPUTER GAMES COMPUTER VISION COMPUTER SOFTWARE MEDICINE NATURAL LANGUAGE PROCESSING 9 8 10 PSYCHOLOGY 9 8 11 RELIABILITY ANALYSIS 9 8 12 SCHEDULING 9 8 13 SPEECH RECOGNITION 9 8 14 VEHICLE CONTROL AND MALFUNCTION DIAGNOSIS 10 LEARNING PROBABILISTIC MODEL PARAMETERS 10 1 LEARNING A SINGLE PARAMETER 10 1 1 BINOMIAL RANDOM VARIABLES 10 1 2 MULTINOMIAL RANDOM VARIABLES 10 2 LEARNING PARAMETERS IN A BAYESIAN NETWORK 10 2 1 PROCEDURE FOR LEARNING PARAMETERS 10 2 2 EQUIVALENT SAMPLE SIZE 10 3 LEARNING PARAMETERS WITH MISSING DATA? 11 LEARNING PROBABILISTIC MODEL STRUCTURE 11 1 STRUCTURE LEARNING PROBLEM 11 2 SCORE-BASED STRUCTURE LEARNING 11 2 1 BAYESIAN SCORE 11 2 2 BIC SCORE 11 2 3 CONSISTENT SCORING CRITERIA 11 2 4 HOW MANY DAGS MUST WE SCORE? 11 2 5 USING THE LEARNED NETWORK TO DO INFERENCE 11 2 6 LEARNING STRUCTURE WITH MISSING DATA? 11 2 7 APPROXIMATE STRUCTURE LEARNING 11 2 8 MODEL AVERAGING 11 2 9 APPROXIMATE MODEL AVERAGING? 11 3 CONSTRAINT-BASED STRUCTURE LEARNING 11 3 1 LEARNING A DAG FAITHFUL TO P 11 3 2 LEARNING A DAG IN WHICH P IS EMBEDDED FAITHFULLY 11 4 APPLICATION: MENTOR 11 4 1 DEVELOPING THE NETWORK 11 4 2 VALIDATING MENTOR 11 5 SOFTWARE PACKAGES FOR LEARNING 11 6 CAUSAL LEARNING 11 6 1 CAUSAL FAITHFULNESS ASSUMPTION 11 6 2 CAUSAL EMBEDDED FAITHFULNESS ASSUMPTION 11 6 3 APPLICATION: COLLEGE STUDENT RETENTION RATE 11 7 CLASS PROBABILITY TREES 11 7 1 THEORY OF CLASS PROBABILITY TREES 11 7 2 APPLICATION TO TARGETED ADVERTISING 11 8 DISCUSSION AND FURTHER READING 11 8 1 BIOLOGY 11 8 2 BUSINESS AND FINANCE 11 8 3 CAUSAL LEARNING 11 8 4 DATA MINING 11 8 5 MEDICINE 11 8 6 WEATHER FORECASTING 12 UNSUPERVISED LEARNING AND REINFORCEMENT LEARNING 12 1 UNSUPERVISED LEARNING 12 1 1 CLUSTERING 12 1 2 AUTOMATED DISCOVERY 12 2 REINFORCEMENT LEARNING 12 2 1 MULTI-ARMED BANDIT ALGORITHMS 12 2 2 DYNAMIC NETWORKS? 12 3 DISCUSSION AND FURTHER READING III EMERGENT INTELLIGENCE 13 EVOLUTIONARY COMPUTATION 13 1 GENETICS REVIEW 13 2 GENETIC ALGORITHMS 13 2 1 ALGORITHM 13 2 2 ILLUSTRATIVE EXAMPLE 13 2 3 TRAVELING SALESPERSON PROBLEM 13 3 GENETIC PROGRAMMING 13 3 1 ILLUSTRATIVE EXAMPLE 13 3 2 ARTI?CIAL ANT 13 3 3 APPLICATION TO FINANCIAL TRADING 13 4 DISCUSSION AND FURTHER READING 14 SWARM INTELLIGENCE 14 1 ANT SYSTEM 14 1 1 REAL ANT COLONIES 14 1 2 ARTI?CIAL ANTS FOR SOLVING THE TSP 14 2 FLOCKS 14 3 DISCUSSION AND FURTHER READING IV NEURAL INTELLIGENCE 15 NEURAL NETWORKS AND DEEP LEARNING 15 1 THE PERCEPTRON 15 1 1 LEARNING THE WEIGHTS FOR A PERCEPTRON 15 1 2 THE PERCEPTRON AND LOGISTIC REGRESSION 15 2 FEEDFORWARD NEURAL NETWORKS 15 2 1 MODELING XOR 15 2 2 EXAMPLE WITH TWO HIDDEN LAYERS 15 2 3 STRUCTURE OF A FEEDFORWARD NEURAL NETWORK 15 3 ACTIVATION FUNCTIONS 15 3 1 OUTPUT NODES 15 3 2 HIDDEN NODES 15 4 APPLICATION TO IMAGE RECOGNITION 15 5 DISCUSSION AND FURTHER READING V LANGUAGE UNDERSTANDING 16 NATURAL LANGUAGE UNDERSTANDING 16 1 PARSING 16 1 1 RECURSIVE PARSER 16 1 2 AMBIGUITY 16 1 3 DYNAMIC PROGRAMMING PARSER 16 1 4 PROBABILISTIC PARSER 16 1 5 OBTAINING PROBABILITIES FOR A PCFG 16 1 6 LEXICALIZED PCFG 16 2 SEMANTIC INTERPRETATION 16 3 CONCEPT/KNOWLEDGE INTERPRETATION 16 4 INFORMATION EXTRACTION 16 4 1 APPLICATIONS OF INFORMATION EXTRACTION 16 4 2 ARCHITECTURE FOR AN INFORMATION EXTRACTION SYSTEM 16 5 DISCUSSION AND FURTHER READING REFERENCES