EFFECTIVE DATABASES FOR TEXT AND DOCUMENT MANAGEMENT
EFFECTIVE DATABASES FOR TEXT & DOCUMENT MANAGEMENT TABLE OF CONTENTS SECTION I: INFORMATION EXTRACTION AND RETRIEVAL IN WEB-BASED SYSTEMS CHAPTER I SYSTEM OF INFORMATION RETRIEVAL IN XML DOCUMENTS CHAPTER II INFORMATION EXTRACTION FROM FREE-TEXT BUSINESS DOCUMENTS CHAPTER III INTERACTIVE INDEXING OF DOCUMENTS WITH A MULTILINGUAL THESAURUS CHAPTER IV MANAGING DOCUMENT TAXONOMIES IN RELATIONAL DATABASES CHAPTER V BUILDING SIGNATURE-TREES ON PATH SIGNATURES IN DOCUMENT DATABASES CHAPTER VI KEYWORD-BASED QUERIES OVER WEB DATABASES CHAPTER VII UNIFYING ACCESS TO HETEROGENEOUS DOCUMENT DATABASES THROUGH CONTEXTUAL METADATA SECTION II: DATA MANAGEMENT AND WEB TECHNOLOGIES CHAPTER VIII DATABASE MANAGEMENT ISSUES IN THE WEB ENVIRONMENT CHAPTER IX APPLYING JAVA-TRIGGERS FOR X-LINK MANAGEMENT IN THE INDUSTRIAL FRAMEWORK SECTION III: ADVANCES IN DATABASE AND SUPPORTING TECHNOLOGIES CHAPTER X METRICS FOR DATA WAREHOUSE QUALITY CHAPTER XI NOVEL INDEXING METHOD OF RELATIONS BETWEEN SALIENT OBJECTS CHAPTER XII A TAXONOMY FOR OBJECT-RELATIONAL QUERIES CHAPTER XIII RE-ENGINEERING AND AUTOMATION OF BUSINESS PROCESSES: CRITERIA FOR SELECTING SUPPORTING TOOLS CHAPTER XIV ACTIVE RULES AND ACTIVE DATABASES: CONCEPTS AND APPLICATIONS SECTION IV: ADVANCES IN RELATIONAL DATABASE THEORY, METHODS AND PRACTICES CHAPTER XV ON THE COMPUTATION OF RECURSION IN RELATIONAL DATABASES CHAPTER XVI UNDERSTANDING FUNCTIONAL DEPENDENCY CHAPTER XVII DEALING WITH RELATIONSHIP CARDINALITY CONSTRAINTS IN RELATIONAL DATABASE DESIGN CHAPTER XVIII REPAIRING AND QUERYING INCONSISTENT DATABASES XI SECTION I INFORMATION EXTRACTION AND RETRIEVAL IN WEB-BASED SYSTEMS CHAPTER I SYSTEM OF INFORMATION RETRIEVAL IN XML DOCUMENTS ABSTRACT THIS CHAPTER INTRODUCES THE PROCESS TO RETRIEVE UNITS (OR SUBDOCUMENTS) OF RELEVANT INFORMATION FROM XML DOCUMENTS FOR THIS, WE USE THE EXTENSIBLE MARKUP LANGUAGE (XML) WHICH IS CONSIDERED AS A NEW STANDARD FOR DATA REPRESENTATION AND EXCHANGE ON THE WEB XML OPENS OPPORTUNITIES TO DEVELOP A NEW GENERATION OF INFORMATION RETRIEVAL SYSTEM (IRS) TO IMPROVE THE INTERROGATION PROCESS OF DOCUMENT BASES ON THE WEB OUR WORK FOCUSES INSTEAD ON END-USERS WHO DO NOT HAVE EXPERTISE IN THE DOMAIN (LIKE A MAJORITY OF THE END-USERS) THIS APPROACH SUPPORTS KEYWORD-BASED SEARCHING LIKE CLASSICAL IRS AND INTEGRATES STRUCTURED SEARCHING WITH THE SEARCH ATTRIBUTES NOTION IT IS BASED ON AN INDEXING METHOD OF DOCUMENT TREE LEAFS WHICH AUTHORIZE A CONTENT- ORIENTED RETRIEVAL THE RETRIEVAL SUBDOCUMENTS ARE RANKED ACCORDING TO THEIR SIMILARITY WITH THE USER’S QUERY WE USE A SIMILARITY MEASURE WHICH IS A COMPROMISE BETWEEN TWO MEASURES: EXHAUSTIVENESS AND SPECIFICITY 2 SMADHI INTRODUCTION THE WORLD WIDE WEB (WWW) CONTAINS LARGE AMOUNTS OF INFORMATION AVAILABLE AT WEBSITES, BUT IT IS DIFFICULT AND COMPLEX TO RETRIEVE PERTINENT INFORMATION INDEED, A LARGE PART OF THIS INFORMATION IS OFTEN STORED AS HYPERTEXT MARKUP LANGUAGE (HTML) PAGES THAT ARE ONLY VIEWED THROUGH A WEB BROWSER THIS RESEARCH IS DEVELOPED IN THE CONTEXT OF THE MEDX PROJECT (LO, 2001) OF OUR TEAM WE USE XML AS A COMMON STRUCTURE FOR STORING, INDEXING AND QUERYING A COLLECTION OF XML DOCUMENTS OUR AIM IS TO PROPOSE THE SUITED SOLUTIONS WHICH ALLOW THE END-USERS NOT SPECIALIZED IN THE DOMAIN TO SEARCH AND EXTRACT PORTIONS OF XML DOCUMENTS (CALLED UNITS OR SUBDOCUMENTS) WHICH SATISFY THEIR QUERIES THE EXTRACTION OF DOCUMENTS PORTION CAN BE REALIZED BY USING XML QUERY LANGUAGES (XQL, XML-QL) (ROBIE, 1999; DEUTSCH, 1999) AN IMPORTANT ASPECT OF OUR APPROACH CONCERNS THE INDEXATION WHICH IS REALIZED ON LEAF ELEMENTS OF THE DOCUMENT TREE AND NOT ON THE WHOLE DOCUMENT KEYWORDS ARE EXTRACTED FROM A DOMAIN THESAURUS A THESAURUS IS A SET OF DESCRIPTORS (OR CONCEPTS) CONNECTED BY HIERARCHICAL RELATIONS, EQUIVALENCE RELATIONS OR ASSOCIATION RELATIONS INDEXING PROCESS RESULTS ARE STORED IN A RESOURCES GLOBAL CATALOG THAT IS EXPLOITED BY THE SEARCH PROCESSOR THIS CHAPTER IS ORGANIZED AS FOLLOWS THE NEXT SECTION DISCUSSES THE PROBLEM OF RELEVANT INFORMATION RETRIEVAL IN THE CONTEXT OF XML DOCUMENTS WE THEN PRESENT THE MODEL OF XML DOCUMENTS INDEXING, FOLLOWED BY THE SIMILARITY MEASURE ADOPTED AND THE RETRIEVAL STRATEGY OF RELEVANT PARTS OF DOCUMENTS THE CHAPTER GOES ON TO DISCUSS RELATED WORK, BEFORE ITS CONCLUSION AN IMPLEMENTATION OF SIRX PROTOTYPE IS CURRENTLY UNDERWAY IN PYTHON LANGUAGE ON LINUX SERVER INFORMATION RETRIEVAL AND XML DOCUMENTS THE CLASSICAL RETRIEVAL INFORMATION INVOLVES TWO PRINCIPAL ISSUES, THE REPRESENTATION OF DOCUMENTS AND QUERIES AND THE CONSTRUCTION OF A RANKING FUNCTION OF DOCUMENTS AMONG INFORMATION RETRIEVAL (IR) MODELS, THE MOST-USED MODELS ARE THE BOOLEAN MODEL, VECTOR SPACE MODEL AND PROBABILIST MODEL IN THE VECTOR SPACE MODEL, DOCU- MENTS AND QUERIES ARE REPRESENTED AS VECTORS IN THE SPACE OF INDEX TERMS DURING THE RETRIEVAL PROCESS, THE QUERY IS ALSO REPRESENTED AS A LIST OF TERMS OR A TERM VECTOR THIS QUERY VECTOR IS MATCHED AGAINST ALL DOCUMENT VECTORS, AND A SIMILARITY MEASURE BETWEEN A DOCUMENT AND A QUERY IS CALCULATED DOCUMENTS ARE RANKED ACCORDING TO THEIR VALUES OF SIMILARITY MEASURE WITH A QUERY XML IS A SUBSET OF THE STANDARD SGML IT HAS A RICHER STRUCTURE THAT IS COMPOSED MAINLY OF AN ELEMENTS TREE THAT FORMS THE CONTENT XML CAN REPRESENT MORE USEFUL INFORMATION ON DATA THAN HTML AN XML DOCUMENT CONTAINS ONLY DATA AS OPPOSED TO AN HTML FILE, WHICH TRIES TO MIX DATA AND PRESENTATION AND USUALLY IGNORES STRUCTURE IT PRESERVES THE STRUCTURE OF THE DATA THAT IT REPRESENTS, WHEREAS HTML FLATTENS IT OUT THIS META MARKUP LANGUAGE DEFINES ITS OWN SYSTEM OF TAGS REPRESENTING THE STRUCTURE OF A DOCUMENT EXPLICITLY HTML PRESENTS INFORMATION AND XML DESCRIBES INFORMATION EFFECTIVE DATABASES FOR TEXT & DOCUMENT MANAGEMENT SYSTEM OF INFORMATION RETRIEVAL IN XML DOCUMENTS A WELL-FORMED XML DOCUMENT DOESN’T IMPOSE ANY RESTRICTIONS ON THE TAGS OR ATTRIBUTE NAMES BUT A DOCUMENT CAN BE ACCOMPANIED BY A DOCUMENT TYPE DEFINITION (DTD), WHICH IS ESSENTIALLY A GRAMMAR FOR RESTRICTING THE TAGS AND STRUCTURE OF A DOCUMENT AN XML DOCUMENT SATISFYING A DTD IS CONSIDERED A VALID DOCUMENT THE DOCUMENT OBJECT MODEL (DOM) IS SIMPLY A SET OF PLANS OR GUIDELINES THAT ENABLES THE USER TO RECONSTRUCT A DOCUMENT RIGHT DOWN TO THE SMALLEST DETAIL THE STRUCTURE OF A DOCUMENT CAN BE TRANSFORMED WITH XSLT (1999) AND ITS CONTENTS DISPLAYED BY USING THE EXTENSIBLE STYLE LANGUAGE (XSL) LANGUAGE OR A PROGRAMMING LANGUAGE (PYTHON, JAVA, ETC ) XSL IS A DECLARATIVE LANGUAGE IN WHICH THE MODEL REFERS THE DATA BY USING PATTERNS IT IS LIMITED WHEN ONE WANTS TO RETRIEVE DATA WITH SPECIFIC CRITERIA, AS ONE CAN REALIZE THAT WITH THE QUERY LANGUAGE XQL (OR OQL) FOR RELATIONAL DATABASES (OR OBJECT) THIS EXTENSION IS PROPOSED BY THREE LANGUAGES COMING FROM THE DATABASE COMMUNITY: XML-QL (FLORESCU, 2000), LOREL (ABITEBOUL, 1997) AND XQL (ROBIE, 1999) FROM THE WEB COMMUNITY REQUIREMENTS FOR A SYSTEM OF RELEVANT INFORMATION RETRIEVAL FOR XML DOCUMENTS WE PROPOSE AN APPROACH FOR INFORMATION RETRIEVAL WITH RELEVANCE RANKING FOR XML DOCUMENTS OF WHICH THE BASIC FUNCTIONAL REQUIREMENTS ARE: A) TO SUPPORT KEYWORD-BASED SEARCHING AND STRUCTURED SEARCHING (BY PROPOSING A SET OF SEARCH ATTRIBUTES) BY END-USERS WHO HAVE NO EXPERTISE IN THE DOMAIN AND OF THAT THE STRUCTURE IS THEN UNKNOWN (LIKE A MAJORITY OF THE END-USERS); B) TO RETRIEVE RELEVANT PARTS OF DOCUMENTS (CALLED SUBDOCUMENTS) RANKED BY THEIR RELEVANCY WITH THE QUERY; AND C) TO NAVIGATE IN THE WHOLE DOCUMENT IN ORDER TO SATISFY THE ESSENTIAL REQUIREMENTS OF THIS APPROACH, WE HAVE OPTED TO: A) USE A DOMAIN THESAURUS; B) DEFINE AN EFFICIENT MODEL OF DOCUMENTS INDEXING THAT EXTENDS THE CLASSIC “INVERTED INDEX” TECHNOLOGY BY INDEXING DOCUMENT STRUCTURE AS WELL AS CONTENT; C) INTEGRATE SEARCH ATTRIBUTES THAT CONCERN A FINITE NUMBER OF SUB-STRUCTURE TYPES, WHICH WE LIKE TO MAKE SEARCHABLE; D) PROPOSE AN INFORMATION RETRIEVAL ENGINE WITH RANKING OF RELEVANT DOCUMENT PARTS ARCHITECTURAL OVERVIEW OF SIRX WE PRESENT AN OVERVIEW OF THE SYSTEM OF INFORMATION RETRIEVAL IN XML DOCUMENTS (SIRX) SHOWING ITS MAINS COMPONENTS (SEE FIGURE 1) THE MAIN ARCHITECTURAL COMPONENTS OF THE SYSTEM ARE THE FOLLOWING: 1) USER INTERFACE: THIS IS USED TO FACILITATE THE INTERACTION BETWEEN THE USER AND THE APPLICATION IT ALLOWS THE USER TO SPECIFY HIS QUERY IT ALSO DISPLAYS RETRIEVED DOCUMENTS OR PARTS OF DOCUMENTS RANKED BY RELEVANCE SCORE IT DOES NOT SUPPOSE AN EXPERTISE OR A DOMAIN KNOWLEDGE OF THE END-USER 2) SEARCH PROCESSOR: THIS ALLOWS RETRIEVAL OF CONTENTS DIRECTLY FROM THE RESOURCES GLOBAL CATALOG ON USING THE VARIOUS INDEX AND KEYWORDS EXPRESSED IN AN INPUT QUERY EFFECTIVE DATABASES FOR TEXT & DOCUMENT MANAGEMENT 4 SMADHI 3) XML DOCUMENTS BASE: THIS STORES XML DOCUMENTS WELL-FORMED IN THEIR ORIGINAL FORMATS 4) THESAURUS: THE DOMAIN THESAURUS CONTAINS THE SET OF DESCRIPTORS (KEYWORDS) WHICH ALLOW THE USER TO INDEX DOCUMENTS OF THIS DOMAIN 5) INDEXER PROCESSOR: FOR EVERY XML DOCUMENT, THE INDEXER PROCESSOR CREATES INDEXES BY USING THE THESAURUS AND THE XML DOCUMENTS BASE THESE INDEXES ALLOW THE USER TO BUILD THE RESOURCES GLOBAL CATALOG 6) RESOURCES GLOBAL CATALOG: THIS IS AN INDEXING STRUCTURE THAT THE SEARCH PROCESSOR USES TO FIND THE RELEVANT DOCUMENT PARTS IT IS EXPLOITED MAINLY BY THE SEARCH PROCESSOR 7) VIEWER: THE VIEWER DISPLAYS RETRIEVED DOCUMENT PARTS THE RESULTS ARE RECOMBINED (XML + XSL) TO SHOW THE DOCUMENT TO THE USER IN AN APPROPRIATE MANNER (INTO HTML) THE MODEL OF XML DOCUMENTS INDEXING IN OUR APPROACH THAT IS BASED ON VECTOR SPACE MODEL, WE PROPOSE TO INDEX THE LEAFS OF THE DOCUMENT TREE (SHIN, 1998) AND THE KEYWORDS THAT CORRESPOND TO THE DESCRIPTOR TERMS EXTRACTED FROM THE DOMAIN THESAURUS (LO, 2000) INDEXING PROCESS RESULTS ARE STRUCTURED BY USING THE XML LANGUAGE IN META-DATA COLLECTION WHICH IS STORED IN THE RESOURCES GLOBAL CATALOG (SEE FIGURE 2) THIS CATALOG IS THE CORE OF THE SIRX SYSTEM IT ENCAPSULATES ALL SEMANTIC CONTENT OF THE XML DOCUMENT’S BASE AND THESAURUS FIGURE 1 THE GENERAL ARCHITECTURE OF SIRX FIGURE 1: THE GENERAL ARCHITECTURE OF SIRX RESULT QUERY USER INTERFACE INDEXER PROCESSOR XML DOCUMENTS BASE THESAURUS SEARCH PROCESSOR VIEWER RESOURCES GLOBAL CATALOG EFFECTIVE DATABASES FOR TEXT & DOCUMENT MANAGEMENT SYSTEM OF INFORMATION RETRIEVAL IN XML DOCUMENTS ELEMENTARY UNITS AND INDEXING IN CLASSIC INFORMATION RETRIEVAL, THE DOCUMENTS ARE CONSIDERED AS ATOMIC UNITS THE KEYWORD SEARCH IS BASED ON CLASSIC INDEX STRUCTURES THAT ARE INVERTED FILES A CLASSIC INVERTED FILE CONTAINS <KEYWORD, DOCUMENT> PAIRS, MEANING THAT THE WORD CAN BE FOUND IN THE DOCUMENT THIS CLASSICAL APPROACH ALLOWS THE USER TO RETRIEVE THE WHOLE DOCUMENT IT IS NOT NECESSARY TO FORGET THAT DOCUMENTS CAN OFTEN BE QUITE LONG AND IN MANY CASES ONLY A SMALL PART OF DOCUMENTS MAY BE RELEVANT TO THE USER’S QUERY IT IS NECESSARY TO BE ABLE TO RETRIEVE ONLY THE PART OF DOCUMENT THAT MAY BE RELEVANT TO THE END-USER’S QUERY TO ACCOMPLISH THIS OBJECTIVE, WE EXTEND THE CLASSIC INVERTED FILE BY MAKING THE UNIT STRUCTURE EXPLICIT THE INDEXING PROCESSOR EXTRACTS TERMS FROM THE THESAURUS AND CALCU- LATES THEIR FREQUENCIES IN EACH ELEMENT AT THE TEXT LEVEL EVERY ELEMENTARY UNIT IS IDENTIFIED IN A UNIQUE WAY BY AN ACCESS-PATH SHOWING HIS POSITION IN THE DOCUMENT THE FORM OF THIS INDEX IS <KEYWORD, UNIT, FREQUENCY> WHERE: 1) KEYWORD IS A TERM APPEARING IN THE CONTENT OF ELEMENT OR VALUES OF AN ATTRIBUTE OF THIS DOCUMENT; 2) UNIT SPECIFIES THE ACCESS PATH TO ELEMENT CONTENT THAT CONTAINS KEYWORD; THE ACCESS PATH IS DESCRIBED BY USING XPATH (1999) COMPLIANCE SYNTAX; 3) FREQUENCY IS THE FREQUENCY OF THE KEYWORD IN THE SPECIFIED UNIT THIS INDEXATION METHOD ALLOWS A DIRECT ACCESS TO ANY ELEMENTARY UNIT WHICH APPEARS IN THE RESULT OF THE QUERY AND REGROUPS RESULTS OF EVERY DOCUMENT BY USING XSLT SEARCH ATTRIBUTES METHODS OF CLASSICAL INFORMATION RETRIEVAL PROPOSE A FUNCTION OF SEARCH FROM SIGNALETIC METADATA (AUTHOR, TITLE, DATE, ETC ) THAT CONCERNS MOSTLY CHARACTERISTICS RELATED TO A WHOLE DOCUMENT TO BE ABLE TO REALIZE SEARCHES ON SUB-STRUCTURES OF A DOCUMENT, WE PROPOSE TO INTEGRATE A SEARCH BASED ON THE DOCUMENT STRUCTURE FROM A FINITE NUMBER OF ELEMENT TYPES, WHICH WE LIKE TO MAKE SEARCHABLE FROM THEIR SEMANTIC CONTENT THESE SPECIFIC ELEMENTS ARE CALLED SEARCH ATTRIBUTES THEY ARE INDEXED LIKE KEYWORDS IN THE RESOURCES GLOBAL CATALOG EVERY SEARCH ATTRIBUTE HAS THE FOLLOWING FORM: <IDENTIFIER, UNIT>, WHERE IDENTIFIER IS THE NAME (OR TAG) OF THE SEARCH ATTRIBUTE UNDER WHICH IT WILL APPEAR TO THE USER, AND UNIT INDICATES THE ACCESS PATH TO A ELEMENTARY UNIT (TYPE 1) OR AN ANOTHER NODE (TYPE 2) OF DOCUMENT THAT WILL CARRY THIS STRUCTURAL SEARCH BASED ON ITS CONTENT SEARCH ATTRIBUTE NAMES ARE AVAILABLE AT THE LEVEL OF THE USER’S INTERFACE IN THE FOLLOWING EXAMPLE, THE TAG OF ELEMENTARY UNIT IS ‘TITLE,’ AND ‘AUTHOR’ IS THE NAME OF AN ATTRIBUTE OF THE TAG ‘BOOK ’ <INFO IDINFO=“TITLE” PATH=“//TITLE”/> <INFO IDINFO=“AUTHOR” PATH=“//BOOK/@AUTHOR”/> THE QUERY RESULT DEPENDS ON TYPE OF SEARCH ATTRIBUTE IF THE INDEXED SEARCH ATTRIBUTE IS AN ELEMENTARY UNIT, THEN THE RETURNED RESULT IS THE NODE THAT IS THE FATHER OF THIS UNIT IF THE INDEXED SEARCH ATTRIBUTE IS A NODE DIFFERENT FROM ELEMENTARY UNIT, THEN THE RETURNED RESULT IS THIS NODE EFFECTIVE DATABASES FOR TEXT & DOCUMENT MANAGEMENT 6 SMADHI QUERY EXAMPLES QUERY 1: TITLE = ’DATAWEB’ THIS QUERY RETURNS THE FOLLOWING RESULT: ALL THE NAMES OF DOCUMENTS OF WHICH VALUE OF <TITLE> CONTAINS ‘DATAWEB’ TEXT QUERY 2: AUTHOR = ’SMADHI’ THIS QUERY RETURNS THE FOLLOWING RESULT: ALL THE SUB-STRUC- TURES (AT FIRST LEVEL) WHICH HAVE FOR NAME ‘BOOK’ AND FOR THAT THE ATTRIBUTE ‘AUTHOR’ CONTAINS ‘SMADHI’ TEXT RESOURCES GLOBAL CATALOG THE RESOURCES GLOBAL CATALOG IS DEFINED AS A GENERALIZED INDEX THAT ALLOWS THE USER TO MAINTAIN FOR SIRX, TO EFFICIENTLY SUPPORT KEYWORD SEARCHING AND SUB-STRUCTURE SEARCHING IT IS USED BY THE SEARCH PROCESSOR USE TO FIND THE RELEVANT DOCUMENTS (OR PARTS OF DOCUMENTS) IT IS REPRESENTED BY AN XML DOCUMENT WHICH DESCRIBES EVERY XML DOCUMENT THAT IS INDEXED BY THE INDEXING PROCESSOR THIS CATALOG IS DESCRIBED IN XML ACCORDING THE FOLLOWING DTD: FIGURE 2 THE CATALOG DTD <!ELEMENT CATALOG(DOC*)> <!ELEMENT DOC(ADDRESS, SEARCH-ATTRIBUTES, KEYWORDS)> <!ATTLIST DOC IDDOC ID #REQUIRED> <!ELEMENT SEARCH-ATTRIBUTES(INFO*)> <!ELEMENT INFO (#PCDATA)> <!ATTLIST INFO IDINFO ID #REQUIRED) <!ATTLIST INFO PATH CDATA #REQUIRED> <!ELEMENT ADDRESS(#PCDTA)> <!ELEMENT KEYWORDS(KEY*)> <!ELEMENT KEY (#PCDATA)> <!ATTLIST KEY IDKEY ID #REQUIRED> <!ATTLIST KEY PATH CDATA #REQUIRED> <!ATTLIST KEY FREQ CDATA #REQUIRED> THE FOLLOWING EXAMPLE ILLUSTRATES THE STRUCTURE OF THIS CATALOG: FIGURE 3 AN EXAMPLE OF RESOURCES GLOBAL CATALOG <CATALOG> <DOC IDDOC=“D1” > <ADDRESS>C:/SRIX/MCSEAI XML</ADDRESS> <SEARCH-ATTRIBUTES> <INFO IDINFO=“TITLE” PATH=“//TITLE”/> <INFO IDINFO=“AUTHOR” PATH=“//BOOK/@AUTHOR”/> </SEARCH-ATTRIBUTES> <KEYWORDS> <KEY IDKEY =“K1” PATH=“//DATAWEB/INTEGRATION” FREQ=2>XML </KEY> <KEY IDKEY =“K2” PATH=“// MAPPING/@BASE” FREQ=1>XML </KEY> EFFECTIVE DATABASES FOR TEXT & DOCUMENT MANAGEMENT SYSTEM OF INFORMATION RETRIEVAL IN XML DOCUMENTS NU: NUMBER OF ELEMENTARY UNITS J OF DOCUMENT I P: NUMBER OF INDEXING KEYWORDS WI JK: WEIGHT OF THE KTH TERM IN THE JTH ELEMENTARY UNIT OF THE ITH DOCUMENT WE USE THE CLASSICAL TF IDF WEIGHTING SCHEME (SALTON, 1988) TO CALCULATE WI JK I JK I JK TF W — K IDF TF I JK: THE FREQUENCY OF THE KTH TERM IN THE JTH ELEMENTARY UNIT OF THE ITH DOCUMENT IDFK: THE INVERSE DOCUMENT FREQUENCY OF THE INDEX TERM TK IT IS COMPUTED AS A FUNCTION OF THE ELEMENTARY UNIT FREQUENCY BY THE FOLLOWING FORMULA: IDFK = LOG(TNU/NUK) TNU: THE TOTAL NUMBER OF ELEMENTARY UNITS IN THE DOCUMENT BASE NUK: THE NUMBER OF ELEMENTARY UNITS WHICH THE KTH TERM OCCURS AT LEAST ONCE EFFECTIVE DATABASES FOR TEXT & DOCUMENT MANAGEMENT 8 SMADHI RELEVANT INFORMATION RETRIEVAL SIRX SUPPORTS TWO WAYS TO RETRIEVE PARTS OF DOCUMENTS: A) QUERYING BY SEARCH ATTRIBUTES: AUTHORIZES A SEARCH BASED ON A DOCUMENT STRUCTURE FROM A LIST OF SEARCH ATTRIBUTES PROPOSED TO A USER IT ALLOWS ONE TO RETRIEVE DOCUMENTS OR PARTS OF DOCUMENTS ACCORDING THE TYPE SEARCH ATTRIBUTES THIS ASPECT IS NOT DETAILED IN THIS CHAPTER B) QUERYING BY CONTENT WITH KEYWORDS: ALLOWS RETRIEVAL OF DOCUMENTS OR PARTS OF DOCUMENTS IN THIS SECTION WE DESCRIBE THE SEARCH PROCESS OF RELEVANT INFORMATION RETRIEVAL THAT INVOLVES TWO ISSUES: GENERATING QUERY VECTOR, AND COMPUTING THE SIMILARITY BETWEEN VECTOR QUERY AND EACH ELEMENTARY UNIT VECTOR THE ADOPTED MODEL OF DATA RESTS MAINLY ON THE USE OF THE CATALOG IN MEMORY CENTRAL FOR AN EXPLOITATION, DURING THE PROCESS OF INTERROGATION BY A SET OF END-USERS QUERY PROCESSING A USER’S QUERY IS A LIST OF ONE OR MORE KEYWORDS WHICH BELONG TO THE THESAURUS WHEN THE USER INPUTS A QUERY, THE SYSTEM GENERATES A QUERY VECTOR BY USING THE SAME INDEXING METHOD AS THAT OF THE ELEMENT UNIT VECTOR A QUERY VECTOR Q IS AS FOLLOWS: Q =(Q1, Q2, …, QK, …, QM) WITH M † P QUERY TERMS QK (J=1…M) ARE WEIGHTED BY THE IDF VALUE WHERE IDF IS MEASURED BY LOG(TNU/NUK) RETRIEVAL AND RANKING OF RELEVANT XML INFORMATION UNITS THE SEARCH PROCESS RETURNS THE RELEVANT ELEMENTARY UNITS OF AN XML DOCUMENT THESE INFORMATION UNITS ARE RANKED ACCORDING TO THEIR SIMILARITY COEFFICIENTS MEASURING THE RELEVANCE OF ELEMENTARY UNITS OF AN XML DOCUMENT TO A USER’S QUERY IN THE VECTOR SPACE MODEL, THIS SIMILARITY IS MEASURED BY COSINE OF THE ANGLE BETWEEN THE ELEMENTARY UNIT VECTOR AND QUERY VECTOR ON CONSIDERING THE TWO VECTORS UI AND Q IN THE EUCLIDEAN SPACE WITH SCALAR PRODUCT NOTED <,> AND NORM NOTED & &, THE SIMILARITY IS (SMADHI, 2001): È È È M J IJ M J J M J IJ J I W Q W Q Q U SIM