THE INTERNATIONAL DICTIONARY OF ARTIFICIAL INTELLIGENCE
WHERE 95% OF THE OUTCOMES BELONG TO ONE CLASS, A MODEL THAT ALWAYS PREDICTS THE MAJORITY CLASS WOULD HAVE A HIGH ACCURACY (95%) BUT FAIL TO IDENTIFY THE MINORITY CLASS IN SUCH CASES, ALTERNATIVE METRICS LIKE PRECISION, RECALL, F1 SCORE, OR AREA UNDER THE ROC CURVE (AUC) PROVIDE MORE INSIGHT INTO THE MODEL’S PERFORMANCE ACCURACY REMAINS A POPULAR METRIC FOR EVALUATING MODELS IN MANY REAL-WORLD APPLICATIONS, BUT ITS LIMITATIONS SHOULD BE CAREFULLY CONSIDERED DEPENDING ON THE PROBLEM DOMAIN 9 ACL SEE AGENT COMMUNICATION LANGUAGE 10 ACO SEE ANT COLONY OPTIMIZATION 11 ACTION IN REINFORCEMENT LEARNING (RL), AN ACTION REFERS TO A DECISION OR MOVE MADE BY AN AGENT WITHIN AN ENVIRONMENT IN RESPONSE TO ITS CURRENT STATE ACTIONS ARE FUNDAMENTAL TO THE LEARNING PROCESS, AS THEY DIRECTLY INFLUENCE THE AGENT’S INTERACTIONS WITH THE ENVIRONMENT AND, SUBSEQUENTLY, THE OUTCOMES IT EXPERIENCES AT EACH TIME STEP, THE AGENT OBSERVES THE STATE OF THE ENVIRONMENT AND SELECTS AN ACTION FROM A PREDEFINED SET OF POSSIBLE ACTIONS, KNOWN AS THE ACTION SPACE THE ACTION SPACE CAN BE DISCRETE (E G , MOVE LEFT, MOVE RIGHT) OR CONTINUOUS (E G , ADJUSTING THE SPEED OF A ROBOTIC ARM) AFTER TAKING AN ACTION, THE ENVIRON- MENT TRANSITIONS TO A NEW STATE AND PROVIDES THE AGENT WITH A REWARD, WHICH THE AGENT USES TO EVALUATE THE EFFECTIVENESS OF ITS CHOSEN ACTION THE AGENT’S GOAL IN RL IS TO LEARN A POLICY, ( )S ? , WHICH DEFINES THE BEST ACTION TO TAKE IN EACH STATE TO MAXIMIZE THE CUMULATIVE REWARD, KNOWN AS THE RETURN, OVER TIME THIS LEARNING IS OFTEN BASED ON TRIAL AND ERROR, WHERE THE AGENT EXPLORES DIFFERENT ACTIONS TO DISCOVER WHICH ONES YIELD THE HIGHEST REWARDS ACTIONS ARE CENTRAL TO THE AGENT’S LEARNING PROCESS, SHAPING THE INTERACTION BETWEEN THE AGENT, ITS ENVIRONMENT, AND THE LONG-TERM GOAL OF OPTIMAL DECISION-MAKING IN REINFORCEMENT LEARNING TASKS 12 ACTIONABLE INSIGHTS ACTIONABLE INSIGHTS IN THE CONTEXT OF EXPLAINABLE AI (XAI) REFER TO THE CLEAR, INTERPRETABLE INFORMATION GENERATED BY AN AI SYSTEM THAT EMPOWERS USERS TO MAKE INFORMED DECISIONS AND TAKE SPECIFIC ACTIONS UNLIKE RAW DATA OR OPAQUE OUTPUTS, ACTIONABLE INSIGHTS PROVIDE MEANINGFUL, UNDERSTANDABLE EXPLANATIONS ABOUT HOW AND WHY THE AI SYSTEM ARRIVED AT A PARTICULAR DECISION OR RECOMMENDATION, ENABLING STAKEHOLDERS TO RESPOND EFFECTIVELY IN XAI, ACTIONABLE INSIGHTS ARE CRUCIAL FOR PROMOTING TRUST, ACCOUNTABILITY, AND USABILITY THESE INSIGHTS ALLOW USERS—WHETHER THEY ARE DATA SCIENTISTS, BUSINESS DECISION- MAKERS, OR END-USERS—TO COMPREHEND THE UNDERLYING PATTERNS AND LOGIC OF AI MODELS FOR EXAMPLE, A MEDICAL AI SYSTEM MIGHT SUGGEST A DIAGNOSIS, BUT ACTIONABLE INSIGHTS EXPLAIN WHICH FEATURES (E G , PATIENT SYMPTOMS, TEST RESULTS) WERE MOST INFLUENTIAL IN THAT DECISION THIS CLARITY ALLOWS MEDICAL PROFESSIONALS TO VALIDATE THE AI’S RECOMMENDATIONS AND ACT ACCORDINGLY MOREOVER, ACTIONABLE INSIGHTS SUPPORT MODEL IMPROVEMENT BY HIGHLIGHTING AREAS OF POTENTIAL BIAS, ERROR, OR MISINTERPRETATION IN HIGH-STAKES APPLICATIONS SUCH AS FINANCE, HEALTHCARE, AND AUTONOMOUS SYSTEMS, THESE INSIGHTS NOT ONLY ENABLE BETTER HUMAN OVERSIGHT BUT ALSO ENSURE THAT AI-DRIVEN DECISIONS ALIGN WITH ETHICAL STANDARDS AND REGULATORY REQUIREMENTS BY CONVERTING COMPLEX AI OUTPUTS INTO UNDERSTANDABLE, CONTEXT-SPECIFIC GUIDANCE, ACTIONABLE INSIGHTS ENHANCE THE OVERALL UTILITY OF EXPLAINABLE AI SYSTEMS 13 ACTION-VALUE FUNCTION IN REINFORCEMENT LEARNING (RL), THE ACTION-VALUE FUNCTION, OFTEN DENOTED AS Q(S, A), REPRESENTS THE EXPECTED CUMULATIVE REWARD AN AGENT CAN OBTAIN BY TAKING A PARTICULAR ACTION A IN A GIVEN STATE S, AND THEN FOLLOWING AN OPTIMAL POLICY THEREAFTER IT IS A CRITICAL CONCEPT IN RL BECAUSE IT QUANTIFIES THE VALUE OF TAKING SPECIFIC ACTIONS IN SPECIFIC STATES, HELPING THE AGENT MAKE INFORMED DECISIONS FORMALLY, THE ACTION-VALUE FUNCTION IS DEFINED AS: ( , ) , , T T T Q S A R S S A A ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? WHERE: S IS THE CURRENT STATE, A IS THE ACTION CHOSEN IN THAT STATE, RT IS THE REWARD RECEIVED AT TIME STEP T, ? IS THE DISCOUNT FACTOR ( ? ? ? 1 ), WHICH CONTROLS THE IMPORTANCE OF FUTURE REWARDS, AND ? IS THE POLICY FOLLOWED BY THE AGENT AFTER TAKING THE ACTION THE ACTION-VALUE FUNCTION IS CLOSELY RELATED TO THE STATE-VALUE FUNCTION V(S), WHICH ONLY CONSIDERS THE EXPECTED RETURN FROM A GIVEN STATE, INDEPENDENT OF THE ACTION TAKEN THE DIFFERENCE IS THAT Q(S, A) EVALUATES BOTH THE STATE AND ACTION PAIR, MAKING IT MORE USEFUL IN LEARNING OPTIMAL POLICIES, ESPECIALLY IN ALGORITHMS LIKE Q-LEARNING IN Q-LEARNING, ONE OF THE MOST POPULAR RL ALGORITHMS, THE AGENT LEARNS AN APPROXIMATION OF THE ACTION-VALUE FUNCTION Q(S, A) BY UPDATING IT ITERATIVELY THROUGH INTERACTIONS WITH THE ENVIRONMENT THE Q-LEARNING UPDATE RULE IS: ( , ) ( , ) MAX ( , ) ( , ) A Q S A Q S A R Q S A Q S A ? ? ? ?? ? ? ? ? ? ? ? ? HERE, ? IS THE LEARNING RATE, R IS THE REWARD, AND S’ IS THE NEW STATE AFTER TAKING ACTION A BY CONTINUOUSLY UPDATING Q(S, A), THE AGENT CONVERGES TOWARDS AN OPTIMAL POLICY THAT MAXIMIZES LONG-TERM REWARDS THE ACTION-VALUE FUNCTION IS CENTRAL TO MANY RL ALGORITHMS, ENABLING AGENTS TO EVALUATE THE POTENTIAL OUTCOMES OF ACTIONS AND MAKE BETTER DECISIONS IN COMPLEX ENVIRONMENTS 14 ACTIVATION FUNCTION AN ACTIVATION FUNCTION IN NEURAL NETWORKS IS A MATHEMATICAL FUNCTION THAT DETERMINES THE OUTPUT OF A NEURON GIVEN ITS INPUT IT INTRODUCES NON-LINEARITY INTO THE NETWORK, ALLOWING IT TO LEARN AND REPRESENT COMPLEX PATTERNS WITHOUT ACTIVATION FUNCTIONS, NEURAL NETWORKS WOULD BEHAVE LIKE LINEAR MODELS, REGARDLESS OF THEIR DEPTH, LIMITING THEIR ABILITY TO SOLVE COMPLEX TASKS FORMALLY, AFTER A NEURON COMPUTES THE WEIGHTED SUM OF ITS INPUTS I I Z W X B ? ? ? , THE ACTIVATION FUNCTION F(Z) TRANSFORMS THIS SUM INTO THE NEURON’S OUTPUT COMMON ACTIVATION FUNCTIONS: 1 SIGMOID: ( ) Z F Z E? ? ? , SQUASHES THE OUTPUT TO A RANGE BETWEEN 0 AND 1, OFTEN USED IN BINARY CLASSIFICATION 2 RELU (RECTIFIED LINEAR UNIT): ( ) MAX( , ) F Z Z ? , OUTPUTS ZERO FOR NEGATIVE INPUTS AND THE INPUT ITSELF FOR POSITIVE VALUES RELU IS WIDELY USED DUE TO ITS SIMPLICITY AND EFFECTIVENESS IN DEEP LEARNING 3 TANH: ( ) TANH( ) F Z Z ? , OUTPUTS VALUES BETWEEN -1 AND 1, OFTEN USED IN HIDDEN LAYERS TO CENTER DATA ACTIVATION FUNCTIONS ARE ESSENTIAL FOR DEEP NETWORKS, ENABLING THEM TO CAPTURE COMPLEX RELATIONSHIPS IN DATA AND PERFORM TASKS LIKE IMAGE RECOGNITION, LANGUAGE TRANSLATION, AND MORE 15 ACTIVE LEARNING ACTIVE LEARNING IS A MACHINE LEARNING TECHNIQUE WHERE THE MODEL SELECTS THE MOST INFORMATIVE DATA POINTS FROM WHICH TO LEARN, REDUCING THE AMOUNT OF LABELED DATA REQUIRED FOR TRAINING THIS APPROACH IS ESPECIALLY USEFUL WHEN OBTAINING LABELED DATA IS EXPENSIVE OR TIME-CONSUMING, AS IN MEDICAL IMAGING OR LEGAL DOCUMENT REVIEW IN ACTIVE LEARNING, THE MODEL INTERACTS WITH AN “ORACLE” (OFTEN A HUMAN EXPERT) TO QUERY LABELS FOR SPECIFIC, UNCERTAIN INSTANCES THAT WILL MOST IMPROVE ITS PERFORMANCE THE MODEL TYPICALLY USES ONE OF THREE QUERY STRATEGIES: 1 UNCERTAINTY SAMPLING: THE MODEL QUERIES THE DATA POINTS FOR WHICH IT IS LEAST CONFIDENT IN ITS PREDICTIONS 2 QUERY BY COMMITTEE: MULTIPLE MODELS (A COMMITTEE) ARE TRAINED, AND THE DATA POINTS WITH THE HIGHEST DISAGREEMENT AMONG THEM ARE SELECTED FOR LABELING 3 EXPECTED MODEL CHANGE: THE MODEL SELECTS INSTANCES THAT ARE EXPECTED TO CAUSE THE MOST SIGNIFICANT CHANGE IN ITS PARAMETERS ACTIVE LEARNING IS ESPECIALLY BENEFICIAL IN SITUATIONS WHERE LABELED DATA IS SPARSE, ENABLING MODELS TO ACHIEVE HIGH ACCURACY WITH FEWER LABELED EXAMPLES COMPARED TO TRADITIONAL PASSIVE LEARNING IT IS COMMONLY APPLIED IN FIELDS LIKE NATURAL LANGUAGE PROCESSING, IMAGE CLASSIFICA- TION, AND MEDICAL DIAGNOSTICS, WHERE LABELED DATA IS COSTLY TO OBTAIN 16 ACTOR-CRITIC THE ACTOR-CRITIC IS A POPULAR REINFORCEMENT LEARNING FRAMEWORK THAT COMBINES THE STRENGTHS OF BOTH VALUE-BASED AND POLICY-BASED METHODS IT IS COMPOSED OF TWO DISTINCT COMPONENTS: THE ACTOR AND THE CRITIC, EACH PLAYING A UNIQUE ROLE IN THE LEARNING PROCESS THE ACTOR IS RESPONSIBLE FOR DETERMINING THE ACTIONS THE AGENT SHOULD TAKE, WHILE THE CRITIC EVALUATES THE ACTIONS BY ESTIMATING THE VALUE FUNCTION, THEREBY GUIDING THE ACTOR TO IMPROVE ITS POLICY COMPONENTS: 1 ACTOR: THE ACTOR REPRESENTS THE POLICY ( ) A S ? ? ? , WHICH IS A MAPPING FROM STATES S TO ACTIONS A THIS POLICY CAN BE EITHER DETERMINISTIC OR STOCHASTIC THE ACTOR’S GOAL IS TO MAXIMIZE THE EXPECTED CUMULATIVE REWARD BY SELECTING ACTIONS IN A WAY THAT IMPROVES THE POLICY OVER TIME IN MATHEMATICAL TERMS, THE ACTOR UPDATES ITS POLICY PARAMETERS ? IN THE DIRECTION THAT INCREASES THE TOTAL REWARD 2 CRITIC: THE CRITIC ESTIMATES THE VALUE OF THE CURRENT POLICY BY APPROXIMATING THE VALUE FUNCTION, OFTEN THE STATE-VALUE FUNCTION V(S) OR THE ACTION-VALUE FUNCTION Q(S, A) THE CRITIC’S JOB IS TO PROVIDE FEEDBACK TO THE ACTOR BY ESTIMATING HOW GOOD A PARTICULAR ACTION TAKEN IN A SPECIFIC STATE WAS THE CRITIC USES TEMPORAL DIFFERENCE (TD) LEARNING TO UPDATE ITS VALUE ESTIMATES BASED ON THE REWARD AND FUTURE STATE TRANSITIONS THE ACTOR-CRITIC METHOD IS TYPICALLY IMPLEMENTED USING TWO NEURAL NETWORKS, ONE FOR THE ACTOR (POLICY NETWORK) AND ONE FOR THE CRITIC (VALUE NETWORK) THE CRITIC HELPS THE ACTOR BY ESTIMATING THE ADVANTAGE OF AN ACTION, WHICH IS THE DIFFERENCE BETWEEN THE EXPECTED RETURN FROM THE CURRENT ACTION AND THE AVERAGE RETURN FROM THE STATE: ( , ) ( , ) ( ) A S A Q S A V S ? ? AT EACH TIME STEP, THE AGENT DOES THE FOLLOWING: 1 ACTOR’S ROLE: THE ACTOR SELECTS AN ACTION AT BASED ON THE CURRENT POLICY ( ) A S ? ? ? 2 CRITIC’S ROLE: THE CRITIC EVALUATES THE ACTION BY CALCULATING THE TEMPORAL DIFFERENCE ERROR ( ) ( ) T T R V S V S ? ? ? ? ? ? , WHERE R IS THE REWARD AND ? IS THE DISCOUNT FACTOR 3 UPDATING: THE ACTOR UPDATES ITS POLICY BASED ON THE CRITIC’S FEEDBACK, AND THE CRITIC UPDATES ITS VALUE ESTIMATES TO BETTER REFLECT FUTURE REWARDS THE ADVANTAGES OF THE METHOD ARE STABLE LEARNING, BECAUSE BY SPLITTING THE ROLES OF THE ACTOR AND CRITIC, THE LEARNING PROCESS BECOMES MORE STABLE COMPARED TO PURE POLICY GRADIENT METHODS, AND THE USE FOR CONTINUOUS ACTION SPACES, WHERE VALUE-BASED METHODS STRUGGLE SEVERAL ADVANCED ALGORITHMS ARE BASED ON THE ACTOR-CRITIC FRAMEWORK, INCLUDING ADVANTAGE ACTOR-CRITIC (A2C) AND PROXIMAL POLICY OPTIMIZATION (PPO), WHICH ENHANCE STABILITY AND PERFORMANCE BY OPTIMIZING THE INTERACTION BETWEEN THE ACTOR AND CRITIC 17 ACT-R ACT-R IS A COGNITIVE ARCHITECTURE, ORIGINALLY DEVELOPED BY JOHN R ANDERSON IN THE 1970S, DESIGNED TO MODEL HUMAN COGNITION BY SIMULATING HOW PEOPLE ORGANIZE KNOWLEDGE AND PRODUCE INTELLIGENT BEHAVIOR THE ARCHITECTURE IS GROUNDED IN PSYCHOLOGICAL THEORY AND AIMS TO UNDERSTAND THE COGNITIVE PROCESSES UNDERLYING HUMAN THOUGHT, LEARNING, AND MEMORY OVER TIME, ACT-R HAS EVOLVED TO BECOME ONE OF THE MOST INFLUENTIAL MODELS IN COGNITIVE SCIENCE, BRIDGING THE GAP BETWEEN NEUROSCIENCE AND ARTIFICIAL INTELLIGENCE BY PROVIDING A COMPUTATIONAL FRAMEWORK THAT MIMICS HOW HUMANS THINK AND LEARN ACT-R IS BASED ON THE IDEA THAT THE MIND CAN BE DIVIDED INTO DIFFERENT MODULES, EACH RESPONSIBLE FOR SPECIFIC COGNITIVE FUNCTIONS THESE MODULES INTERACT TO PERFORM TASKS AND PROCESS INFORMATION THE ARCHITECTURE DIVIDES COGNITION INTO DECLARATIVE MEMORY, PROCEDURAL MEMORY, AND SEVERAL PERCEPTUAL-MOTOR MODULES: 1 DECLARATIVE MEMORY: THIS MODULE STORES FACTUAL KNOWLEDGE IN THE FORM OF CHUNKS, WHICH ARE STRUCTURED DATA UNITS THAT REPRESENT INFORMATION, SUCH AS CONCEPTS OR EXPERIENCES EACH CHUNK CONTAINS SLOTS AND VALUES (E G , A CHUNK FOR A BIRD MIGHT INCLUDE SLOTS FOR WINGS, FEATHERS, AND THE ABILITY TO FLY) 2 PROCEDURAL MEMORY: THIS CONTAINS PRODUCTION RULES, WHICH ARE CONDITION-ACTION PAIRS USED TO DETERMINE BEHAVIOR WHEN A CERTAIN CONDITION IS MET, A PRODUCTION RULE FIRES, LEADING TO AN ACTION OR THOUGHT PROCESS PROCEDURAL KNOWLEDGE, UNLIKE DECLARATIVE MEMORY, IS KNOWLEDGE OF HOW TO DO SOMETHING (E G , RIDING A BIKE OR SOLVING A MATH PROBLEM) 3 PERCEPTUAL-MOTOR MODULES: ACT-R INTEGRATES PERCEPTUAL MODULES FOR VISION AND HEARING AND MOTOR MODULES FOR CONTROLLING ACTIONS LIKE SPEAKING OR MOVING THESE MODULES ALLOW THE ARCHITECTURE TO INTERACT WITH THE ENVIRONMENT IN A WAY SIMILAR TO HUMANS ACT-R OPERATES IN A CYCLE WHERE IT RETRIEVES KNOWLEDGE FROM DECLARATIVE MEMORY, EVALUATES PROCEDURAL RULES, AND SELECTS THE MOST APPROPRIATE ACTION BASED ON THE CURRENT SITUATION TWO KEY COMPONENTS FACILITATE THIS PROCESS: - GOAL MODULE: THE GOAL MODULE KEEPS TRACK OF THE CURRENT TASK OR OBJECTIVE THE SYSTEM IS WORKING TOWARD IT MAINTAINS THE FOCUS OF COGNITIVE PROCESSES AND ENSURES THAT ACTIONS ARE ALIGNED WITH THE OVERALL GOAL - BUFFERS: BUFFERS ARE TEMPORARY MEMORY SLOTS THAT STORE INFORMATION BEING PROCESSED OR MANIPULATED IN REAL-TIME EACH MODULE HAS ITS OWN BUFFER THAT COMMUNICATES WITH THE CENTRAL PRODUCTION SYSTEM, ALLOWING THE ARCHITECTURE TO FUNCTION IN A FLEXIBLE, REAL-TIME MANNER ACT-R INCORPORATES SEVERAL MECHANISMS TO SIMULATE LEARNING AND MEMORY: 1 CHUNK LEARNING: AS THE SYSTEM PROCESSES INFORMATION, NEW CHUNKS ARE CREATED AND STORED IN DECLARATIVE MEMORY OVER TIME, THESE CHUNKS BECOME MORE ACCESSIBLE AS THEY ARE USED FREQUENTLY, REFLECTING HOW HUMAN MEMORY STRENGTHENS WITH REPETITION 2 PRODUCTION RULE LEARNING: ACT-R CAN MODIFY AND OPTIMIZE ITS PRODUCTION RULES THROUGH A PROCESS KNOWN AS UTILITY LEARNING THE SYSTEM KEEPS TRACK OF THE SUCCESS OF EACH RULE IN ACHIEVING ITS GOALS AND ADJUSTS ITS BEHAVIOR BASED ON THE PAST EFFECTIVENESS OF RULES THIS REFLECTS HOW HUMANS REFINE THEIR SKILLS AND KNOWLEDGE THROUGH EXPERIENCE 3 REINFORCEMENT LEARNING: ACT-R CAN ALSO INCORPORATE REINFORCEMENT LEARNING TO ADJUST DECISION-MAKING BASED ON REWARDS OR PUNISHMENTS THIS ALLOWS IT TO DYNAMICALLY ADAPT ITS STRATEGIES BASED ON THE OUTCOMES OF PREVIOUS ACTIONS ACT-R HAS BEEN USED TO MODEL A WIDE VARIETY OF HUMAN COGNITIVE PROCESSES, FROM SIMPLE REACTION-TIME TASKS TO MORE COMPLEX BEHAVIORS LIKE LEARNING A LANGUAGE OR SOLVING ALGEBRAIC PROBLEMS SOME NOTABLE APPLICATIONS INCLUDE: - HUMAN-COMPUTER INTERACTION: ACT-R HAS BEEN USED TO PREDICT HOW PEOPLE INTERACT WITH COMPUTERS, INCLUDING THE TIME IT TAKES TO PERFORM TASKS LIKE TYPING, CLICKING, OR NAVIGATING INTERFACES - EDUCATIONAL TECHNOLOGIES: ACT-R’S MODELS OF LEARNING AND MEMORY HAVE BEEN APPLIED IN INTELLIGENT TUTORING SYSTEMS TO HELP STUDENTS LEARN MORE EFFECTIVELY BY ADAPTING INSTRUCTIONAL STRATEGIES BASED ON COGNITIVE PRINCIPLES - PSYCHOLOGICAL AND NEUROSCIENTIFIC RESEARCH: ACT-R IS USED TO SIMULATE COGNITIVE TASKS TO BETTER UNDERSTAND HOW THE BRAIN SUPPORTS THOUGHT AND BEHAVIOR BY MAPPING SPECIFIC MODULES IN THE ARCHITECTURE TO BRAIN REGIONS, RESEARCHERS USE ACT-R AS A TOOL FOR EXPLORING NEURAL CORRELATES OF COGNITION ONE OF ACT-R’S KEY CONTRIBUTIONS IS ITS ATTEMPT TO MAP ITS MODULES TO SPECIFIC REGIONS OF THE BRAIN FOR EXAMPLE, THE DECLARATIVE MEMORY MODULE CORRESPONDS TO THE HIPPOCAMPUS, WHILE THE GOAL MODULE IS LINKED TO THE PREFRONTAL CORTEX THIS ALIGNMENT ALLOWS RESEARCHERS TO TEST ACT-R MODELS AGAINST EMPIRICAL NEUROSCIENCE DATA, MAKING IT A POWERFUL FRAMEWORK FOR BRIDGING COMPUTATIONAL MODELS WITH BIOLOGICAL THEORIES OF THE BRAIN 18 ADABOOST ADABOOST, SHORT FOR ADAPTIVE BOOSTING, IS AN ENSEMBLE LEARNING ALGORITHM THAT CREATES A STRONG CLASSIFIER BY COMBINING MULTIPLE WEAK CLASSIFIERS IT WAS INTRODUCED BY YOAV FREUND AND ROBERT SCHAPIRE IN 1995 AND IS WIDELY USED FOR CLASSIFICATION TASKS ADABOOST WORKS PARTICULARLY WELL WHEN USED WITH WEAK LEARNERS LIKE DECISION STUMPS (I E , DECISION TREES WITH A SINGLE SPLIT), BUT IT CAN BE APPLIED TO OTHER CLASSIFIERS AS WELL THE MAIN IDEA OF ADABOOST IS TO ITERATIVELY TRAIN WEAK CLASSIFIERS, WITH EACH CLASSIFIER FOCUSING ON THE MISTAKES MADE BY THE PREVIOUS ONE IT ADJUSTS THE WEIGHTS OF THE TRAINING DATA POINTS SO THAT MISCLASSIFIED POINTS ARE GIVEN MORE IMPORTANCE IN THE NEXT ITERATION THIS PROCESS CONTINUES UNTIL THE ENSEMBLE OF WEAK LEARNERS FORMS A STRONG CLASSIFIER THAT ACHIEVES HIGHER ACCURACY STEPS: 1 INITIALIZE WEIGHTS: AT THE START, EACH TRAINING DATA POINT IS ASSIGNED AN EQUAL WEIGHT IF THERE ARE N DATA POINTS, THE WEIGHT FOR EACH POINT I IS I W N ?1 2 TRAIN WEAK LEARNER: A WEAK LEARNER (E G , A DECISION STUMP) IS TRAINED ON THE DATASET WITH THE WEIGHTED EXAMPLES THE WEAK LEARNER MINIMIZES THE WEIGHTED CLASSIFICATION ERROR 3 COMPUTE CLASSIFIER WEIGHT: FOR EACH WEAK CLASSIFIER ( ) TH X , ITS WEIGHT T ? IS COMPUTED BASED ON THE WEIGHTED ERROR T IT MAKES: LN T T T ? ? ? ? ? ? ? ? ? WHERE T IS THE ERROR RATE OF THE WEAK CLASSIFIER ON THE WEIGHTED DATASET THIS WEIGHT REFLECTS THE CLASSIFIER’S CONTRIBUTION TO THE FINAL MODEL, WITH MORE ACCURATE CLASSIFIERS RECEIVING HIGHER WEIGHTS 4 UPDATE WEIGHTS: THE WEIGHTS OF MISCLASSIFIED DATA POINTS ARE INCREASED, MAKING THE ALGORITHM FOCUS MORE ON THE HARDER-TO-CLASSIFY POINTS IN THE NEXT ITERATION THE WEIGHT UPDATE RULE FOR EACH DATA POINT I IS: ? ? ( ) ( ) EXP { ( ) } T T I I T T I I W W I H X Y ? ? ? ? ? ? , WHERE I IS THE INDICATOR FUNCTION THAT EQUALS 1 IF THE CLASSIFIER MISCLASSIFIES THE POINT XI, AND YI IS THE TRUE LABEL 5 FINAL STRONG CLASSIFIER: AFTER T ITERATIONS, THE FINAL STRONG CLASSIFIER H(X) IS A WEIGHTED SUM OF THE WEAK CLASSIFIERS: ( ) SIGN ( ) T T T T H X H X ? ? ? ? ? ? ? ? ? ? THIS IS A LINEAR COMBINATION OF THE WEAK LEARNERS, WHERE T ? REPRESENTS THE WEIGHT OF EACH WEAK LEARNER BASED ON ITS ACCURACY STRENGTHS: - ADAPTIVE: FOCUSES ON HARDER EXAMPLES, IMPROVING ACCURACY OVER ITERATIONS - SIMPLE: WORKS WELL WITH SIMPLE CLASSIFIERS LIKE DECISION STUMPS - ROBUST TO OVERFITTING: PERFORMS WELL EVEN WHEN TRAINED ON RELATIVELY SMALL DATASETS ADABOOST HAS BEEN WIDELY USED IN VARIOUS DOMAINS, SUCH AS OBJECT DETECTION AND FACE RECOGNITION, DUE TO ITS ABILITY TO BOOST WEAK CLASSIFIERS AND CREATE A ROBUST FINAL MODEL 19 ADAGRAD ADAGRAD (ADAPTIVE GRADIENT ALGORITHM) IS AN OPTIMIZATION ALGORITHM USED TO TRAIN MACHINE LEARNING MODELS, PARTICULARLY DEEP NEURAL NETWORKS IT MODIFIES THE LEARNING RATE DYNAMICALLY DURING TRAINING BY ADAPTING IT FOR EACH PARAMETER BASED ON THE HISTORICAL GRADIENTS OF THE LOSS FUNCTION WITH RESPECT TO THAT PARAMETER THIS MAKES ADAGRAD PARTICULARLY EFFECTIVE FOR DEALING WITH SPARSE DATA AND FEATURES, AS IT NATURALLY ADJUSTS TO THE FREQUENCY OF FEATURES IN THE DATA IN ADAGRAD, PARAMETERS THAT HAVE RECEIVED LARGE UPDATES IN THE PAST ARE SCALED DOWN WITH SMALLER LEARNING RATES, WHILE PARAMETERS THAT HAVE RECEIVED SMALLER UPDATES ARE SCALED UP, ALLOWING FOR MORE FLEXIBILITY IN TRAINING THE UPDATE RULE FOR ADAGRAD INVOLVES MAINTAINING A CUMULATIVE SUM OF SQUARED GRADIENTS, WHICH IS THEN USED TO ADJUST THE LEARNING RATE FOR EACH PARAMETER THE FORMULA FOR THE PARAMETER UPDATE IS: T T T T G G ? ? ? ?? ? ? ? WHERE GT IS THE SUM OF SQUARES OF PAST GRADIENTS, ? IS THE LEARNING RATE, GT IS THE CURRENT GRADIENT, AND IS A SMALL VALUE ADDED FOR NUMERICAL STABILITY ONE LIMITATION OF ADAGRAD IS THAT THE ACCUMULATED SQUARED GRADIENTS CONTINUE GROWING, LEADING TO A PROGRESSIVELY SMALLER LEARNING RATE, WHICH CAN CAUSE THE ALGORITHM TO STOP LEARNING TOO EARLY DESPITE THIS, ADAGRAD IS WIDELY USED IN TEXT CLASSIFICATION AND NATURAL LANGUAGE PROCESSING TASKS 20 ADALINE ADALINE, SHORT FOR ADAPTIVE LINEAR NEURON, IS AN EARLY NEURAL NETWORK MODEL DEVELOPED BY BERNARD WIDROW AND TED HOFF IN THE 1960S IT IS A SINGLE-LAYER NEURAL NETWORK THAT USES A LINEAR ACTIVATION FUNCTION FOR THE OUTPUT AND IS SIMILAR TO THE PERCEPTRON BUT WITH A CRUCIAL DIFFERENCE: INSTEAD OF USING A STEP FUNCTION FOR CLASSIFICATION, ADALINE COMPUTES THE WEIGHTED SUM OF INPUTS AND APPLIES A LINEAR OUTPUT BEFORE THE ACTIVATION IN ADALINE, THE OUTPUT IS GIVEN BY: N N Y W X W X W X B B ? ? ??? ? ? ? W X , WHERE: WI ARE THE WEIGHTS, XI ARE THE INPUT FEATURES, AND B IS THE BIAS TERM THE OUTPUT IS COMPARED TO THE TRUE LABEL, AND THE ERROR IS COMPUTED AS THE DIFFERENCE BETWEEN THE ACTUAL OUTPUT AND THE PREDICTED ONE ADALINE USES THE MEAN SQUARED ERROR (MSE) AS THE LOSS FUNCTION AND UPDATES THE WEIGHTS USING GRADIENT DESCENT THE WEIGHT UPDATE RULE IN ADALINE IS: ˆ ( ) I I I W W Y Y X ? ? ? ? , WHERE ? IS THE LEARNING RATE, Y IS THE TRUE LABEL, AND ˆY IS THE PREDICTED OUTPUT ADALINE PAVED THE WAY FOR MORE ADVANCED NEURAL NETWORKS, CONTRIBUTING TO THE DEVELOPMENT OF BACKPROPAGATION AND MULTI-LAYER NEURAL NETWORKS DESPITE ITS SIMPLICITY, IT DEMONSTRATED THE POWER OF GRADIENT-BASED LEARNING FOR NEURAL MODELS 21 ADAM THE ADAM OPTIMIZER IS A POPULAR OPTIMIZATION ALGORITHM IN MACHINE LEARNING AND DEEP LEARNING, INTRODUCED BY DIEDERIK P KINGMA AND JIMMY BA IN 2014 IT COMBINES THE ADVANTAGES OF TWO OTHER METHODS—ADAGRAD (ADAPTIVE GRADIENT ALGORITHM) AND RMSPROP (ROOT MEAN SQUARE PROPAGATION)—TO PROVIDE EFFICIENT AND EFFECTIVE GRADIENT-BASED OPTIMIZATION ADAM IS PARTICULARLY WELL-SUITED FOR PROBLEMS WITH LARGE DATASETS AND HIGH- DIMENSIONAL PARAMETER SPACES, AS COMMONLY ENCOUNTERED IN DEEP LEARNING KEY FEATURES: 1 ADAPTIVE LEARNING RATES: ADAM ADJUSTS THE LEARNING RATE FOR EACH PARAMETER INDIVIDUALLY BASED ON THE ESTIMATES OF FIRST AND SECOND MOMENTS (MEAN AND UNCENTERED VARIANCE) OF THE GRADIENTS THIS ALLOWS FOR FINE-TUNED ADJUSTMENTS IN THE LEARNING PROCESS, PARTICULARLY FOR PARAMETERS WITH DIFFERENT LEVELS OF SENSITIVITY 2 MOMENTUM AND ADAPTIVE SCALING: ADAM INCORPORATES MOMENTUM BY KEEPING TRACK OF BOTH THE FIRST MOMENT (THE MEAN OF GRADIENTS) AND THE SECOND MOMENT (THE UNCENTERED VARIANCE OF GRADIENTS) THIS HELPS IN ACCELERATING CONVERGENCE AND SMOOTHING THE OPTIMIZATION PROCESS BY REDUCING OSCILLATIONS ALGORITHM: AT EACH ITERATION T, ADAM UPDATES THE MODEL PARAMETERS ? BASED ON THE FOLLOWING STEPS: 1 COMPUTE THE GRADIENT OF THE LOSS FUNCTION WITH RESPECT TO PARAMETERS GT 2 UPDATE BIASED ESTIMATES OF FIRST AND SECOND MOMENTS: ( ) (FIRST MOMENT) T T T M M G ? ? ? ? ? ? ( ) (SECOND MOMENT) T T T V V G ? ? ? ? ? ? 3 BIAS-CORRECTED ESTIMATES: ˆ ˆ , T T T T T T M V M V ? ? ? ? ? ? 4 PARAMETER UPDATE RULE: ˆ ˆ T T T T M V ? ? ? ?? ? ? WHERE: ? IS THE LEARNING RATE, ?1 AND ?2 ARE DECAY RATES FOR FIRST AND SECOND MOMENTS (TYPICALLY 0 9 AND 0 999), AND IS A SMALL CONSTANT TO AVOID DIVISION BY ZERO ADVANTAGES: - EFFICIENCY: WORKS WELL WITH LARGE DATASETS AND SPARSE GRADIENTS - ROBUSTNESS: HANDLES NOISY DATA AND NON-STATIONARY OBJECTIVES EFFECTIVELY - FAST CONVERGENCE: OFTEN CONVERGES FASTER THAN OTHER METHODS, MAKING IT SUITABLE FOR DEEP LEARNING MODELS ADAM HAS BECOME A DEFAULT OPTIMIZER FOR MANY NEURAL NETWORKS DUE TO ITS BALANCE BETWEEN SIMPLICITY, SPEED, AND PERFORMANCE IN PRACTICE 22 ADAPTIVE LEARNING RATE AN ADAPTIVE LEARNING RATE IS A TECHNIQUE USED IN THE BACKPROPA- GATION ALGORITHM TO DYNAMICALLY ADJUST THE LEARNING RATE DURING THE TRAINING OF A NEURAL NETWORK UNLIKE A FIXED LEARNING RATE, WHICH REMAINS CONSTANT THROUGHOUT THE TRAINING PROCESS, ADAPTIVE LEARNING RATES MODIFY THE STEP SIZE BASED ON THE CHARACTERISTICS OF THE GRADIENT DURING TRAINING, ALLOWING THE NETWORK TO CONVERGE MORE EFFICIENTLY IN BACKPROPAGA- TION, THE LEARNING RATE ? CONTROLS HOW MUCH THE WEIGHTS ARE UPDATED WITH RESPECT TO THE CALCULATED GRADIENTS A FIXED LEARNING RATE CAN BE PROBLEMATIC: IF IT IS TOO LARGE, THE NETWORK MAY OVERSHOOT THE OPTIMAL POINT; IF IT IS TOO SMALL, THE TRAINING PROCESS CAN BE SLOW ADAPTIVE LEARNING RATE METHODS ADJUST ? TO IMPROVE TRAINING KEY METHODS: 1 ADAGRAD: ADAPTS THE LEARNING RATE FOR EACH PARAMETER BASED ON THE CUMULATIVE SUM OF PAST GRADIENTS, REDUCING THE LEARNING RATE OVER TIME FOR FREQUENTLY UPDATED PARAMETERS 2 RMSPROP: MODIFIES ADAGRAD BY INTRODUCING A DECAY FACTOR THAT PREVENTS THE LEARNING RATE FROM SHRINKING TOO MUCH 3 ADAM: COMBINES IDEAS FROM ADAGRAD AND RMSPROP, ADJUSTING LEARNING RATES USING BOTH THE FIRST AND SECOND MOMENTS OF THE GRADIENT BY ADAPTING THE LEARNING RATE DURING TRAINING, THESE METHODS IMPROVE CONVERGENCE, ESPECIALLY IN DEEP LEARNING MODELS WHERE DIFFERENT PARAMETERS MAY REQUIRE DIFFERENT LEARNING RATES AT DIFFERENT STAGES OF TRAINING 23 ADAPTIVE RESONANCE THEORY ADAPTIVE RESONANCE THEORY (ART) IS A NEURAL NETWORK MODEL DEVELOPED BY STEPHEN GROSSBERG IN 1976 TO ADDRESS THE STABILITY-PLASTICITY DILEMMA IN LEARNING THIS DILEMMA ARISES FROM THE CHALLENGE OF BALANCING THE ABILITY TO LEARN NEW INFORMATION (PLASTICITY) WHILE RETAINING PREVIOUSLY LEARNED KNOWLEDGE (STABILITY) ART NETWORKS ARE DESIGNED TO ALLOW CONTINUAL LEARNING OF NEW PATTERNS WITHOUT FORGETTING PREVIOUSLY LEARNED PATTERNS, MAKING THEM HIGHLY SUITED FOR TASKS INVOLVING ONLINE LEARNING AND PATTERN RECOGNITION CORE PRINCIPLES: 1 RESONANCE: ART NETWORKS ARE BASED ON THE IDEA OF RESONANCE, WHERE A STABLE STATE IS REACHED WHEN THE INPUT MATCHES A PATTERN STORED IN THE NETWORK IF THE INPUT SUFFICIENTLY MATCHES A LEARNED CATEGORY, THE NETWORK RESONATES, REINFORCING THAT CATEGORY’S REPRESENTA- TION THIS IS AKIN TO A CONFIRMATION PROCESS WHERE THE NETWORK “RESONATES” WITH FAMILIAR INPUTS 2 FAST LEARNING: ART NETWORKS ARE CAPABLE OF FAST, ONE-SHOT LEARNING WHEN AN INPUT DOES NOT RESONATE WITH ANY EXISTING CATEGORY, THE NETWORK QUICKLY FORMS A NEW CATEGORY TO REPRESENT THE NOVEL INPUT 3 VIGILANCE PARAMETER: A KEY FEATURE OF ART IS THE VIGILANCE PARAMETER, WHICH CONTROLS HOW CLOSELY AN INPUT MUST MATCH AN EXISTING CATEGORY FOR THE NETWORK TO RESONATE IF THE VIGILANCE IS HIGH, THE NETWORK REQUIRES A CLOSE MATCH BEFORE CLASSIFYING AN INPUT AS A KNOWN PATTERN,