Basic concepts of computer lexicography

Размещено на http://www.allbest.ru/

Basic concepts of computer lexicography

Olha SERDIUKOVA,

Senior Lecturer at the Department of Foreign Languages National Technical University “Kharkiv Polytechnic Institute” (Kharkiv, Ukraine)

Olga LAZAREVA,

Candidate of Technical Sciences, Associate Professor at the Department of Foreign Languages National Technical University “Kharkiv Polytechnic Institute” (Kharkiv, Ukraine)

Inna CHERNIAVSKA,

Senior Lecturer at the Department of Management, Business and Professional Communication V.N. Karazin Kharkiv National University (Kharkiv, Ukraine)

The article is devoted to researching computer lexicography and aims at regarding basic concepts of the applied scientific discipline in linguistics. Such tasks of computer lexicography, as parsing, creating a lexical database, lexical analysis of electronic dictionaries, estimation of the number of multi-valued-single-valued words, automatic extraction of hyponym-hypernym relationships, displaying values when extracting from several dictionaries at once, defining a value within a single dictionary and extracting information using a set of monolingual and translation dictionaries are discussed. The major principles of a computer dictionary compiling and an example of a dictionary entry are provided. Such terms as lexeme, lemma and lemmatization are explained. Corpus lexicography as a relatively new branch of computer lexicography is overviewed. The authors also note that computational linguistics is closely related to the central problem of artificial intelligence. It is emphasized that although the aims of traditional and computer lexicography are mainly the same, their methods, tools and approaches differ significantly. It is shown that such features of electronic dictionaries as multifunctionality, the use of multimedia, possibility of updating, convenient search and many others provide considerable advantages over conventional paper dictionaries. Practical lexicography is defined as the process of compiling dictionaries of various types on the basis of theoretical developments. The process of creating electronic dictionaries is discussed. Depending on the purpose of the dictionary, its volume, the order of the words in it, the object of description and other criteria, dictionaries can be divided into various types. Such dictionary types as explanatory, translation, etymological, frequency etc. are described. It is concluded that for researchers working in the sphere of computer lexicography and otherfields ofapplied linguistics, understanding ofthe basic principles, tasks andfunctionality of computer-based dictionaries is vital.

Key words: computer lexicography, hypertext, parsing, information extraction, hyponym-hypernym relationships, corpus lexicography, cyber lexicography.

ОСНОВНІ КОНЦЕПТИ КОМП'ЮТЕРНОЇ ЛЕКСИКОГРАФІЇ

Ольга СЕРДЮКОВА,

старший викладач кафедри іноземних мов Національного технічного університету «Харківський політехнічний інститут» (Харків, Україна)

Ольга ЛАЗАРЄВА,

кандидат технічних наук, доцент кафедри іноземних мов Національного технічного університету «Харківський політехнічний інститут» (Харків, Україна)

Інна ЧЕРНЯВСЬКА,

старший викладач кафедри менеджменту, бізнесу та професійних комунікацій Харківського національного університету імені В.Н. Каразіна (Харків, Україна) computer lexicography linguistic

Стаття присвячена дослідженню комп'ютерної лексикографії з метою огляду базових концептів цієї прикладної наукової дисципліни в лінгвістиці. Розглядаються такі завдання комп'ютерної лексикографії як синтаксичний аналіз, створення лексичної бази даних, лексичний аналіз електронних словників, визначення кількості багатозначних та однозначних слів, автоматичне виділення відношень гіпонім-гіперонім, подання визначень при одночасному виборі з декількох словників, визначення лексеми в межах одного словника та відбір інформації з використанням одномовного та перекладних словників. Надано основні принципи компіляції комп'ютерних словників та приклад словникової статті. Дається пояснення таким термінам як лексема, лема та лематизація. Коротко розглянуто корпусну лексикографію як відносно нову галузь комп'ютерної лексикографії. Автори також зазначають, що комп'ютерна лексикографія тісно пов'язана з центральною проблемою штучного інтелекту. Підкреслюється, що хоча цілі традиційної та комп'ютерної лексикографії практично однакові, їхні методи, засоби та підходи суттєво відрізняються. Показано, що такі особливості електронних словників як мультифункціональність, використання мультимедіа, можливості оновлення, зручний пошук та багато інших надають їм суттєву перевагу порівняно з традиційними паперовими словниками. Практична лексикографія визначена як процес компіляції словників різних типів на основі теоретичних напрацювань. Розглядається процес створення електронних словників. Залежно від призначення словника, його обсягу, порядку слів у ньому, об'єкта опису та інших критеріїв, словники можуть поділятись на різні типи. Розглянуто такі типи словників як тлумачні, перекладні, етимологічні, частотні та інші. Робиться висновок, що для дослідників, що працюють у сфері комп'ютерної лексикографії, суттєвим є розуміння основних принципів, задач та функціональності комп'ютерних словників.

Ключові слова: комп'ютерна лексикографія, гіпертекст, синтаксичний аналіз, відбір інформації, відношення гіпонім-гіперонім, корпусна лексикографія, кібер-лексикографія.

Problem statement. Computer lexicography is an applied scientific discipline in linguistics that studies the methods of using computer technology to compile dictionaries. This is a temporary discipline of the period of transition from manual and handwritten lexicographic practice to new paperless information technologies. Within the framework of computer lexicography, computer technologies for the compilation and operation of dictionaries are being developed. Special programs - databases, computer file cabinets, text-processing programs - allow you to automatically generate dictionary entries, store dictionary information and process it. Computer lexicographic programs are divided into two large groups: programs for supporting lexicographic work and electronic dictionaries. (Amsler, 1982: 657-663)

Initially, electronic dictionaries had the same notation as conventional dictionaries, and researchers had to spend a lot of time interpreting this notation, e.g. to determine which part of speech a certain word belongs to. With the development of technology, publishers have decided to separate the database of an electronic dictionary from how it looks when printed. Today, there are more convenient forms of notation, such as extensible XML markup language. Using XML, researchers get quick access to the information stored in the electronic dictionary.

The aim of the article is to give an overview and clarification of the variety of concepts, terms and tasks within computer lexicography.

Basic concepts of computer lexicography. An automatic dictionary is a dictionary in a special machine format designed for use on a computer by a user or a computer word processing program. In other words, a distinction is made between auto-matic human user dictionaries and automatic dictionaries for word processing programs. Automatic dictionaries intended for humans, in terms of interface and structure of a dictionary entry, differ significantly from automatic dictionaries included in machine translation systems, automatic referencing systems, information retrieval systems etc.

Hypertext is a set of texts with relationships connecting them (transition system). Hypertext technologies make it easy to combine different types of information - plain text, drawing, graph, table, diagram, sound and moving image. Both traditional text and hypertext are phenomena generated by new technologies. The practical value of hypertext in computer lexicography lies in the fact that it describes a type of interactive environment with the ability to follow links. The non-linear nature of hypertext makes it possible to arrange information in the form of a branched structure, which makes it possible to significantly expand the scope of the dictionary entry, i. e. it describes the type of interactive environment with the ability to follow links. Words, phrases, or pictures that are links to a particular text or picture allow the user to select the relevant information and view related information.

Dictionary navigation tools are links embedded in various elements of the electronic environment, a part of the hypertext device of an electronic dictionary, which is a combination of a semantic structure, a structure of internal links of a certain content and a technical environment and technical means that give a person an opportunity to master the structure of semantic connections, as well as to make transitions between interconnected elements.

Electronic dictionaries. The term electronic dictionary can be used to refer to any reference material stored electronically that provides information about the spelling, meaning, or use of words. Thus, a spell checker in a text editor, a device that scans and translates printed words, and an electronic version of a paper dictionary are all electronic dictionaries with similar storage and retrieval systems.

In (Nesi, 2000: 839-847), several categories of electronic dictionaries for language learning are dis-tinguished: Internet dictionaries, glossaries for online learning courses, CD-ROM dictionaries, and pocket electronic dictionaries. Nesi lists some of the most famous dictionaries on the CD: Collins Cobuild Stu-dent's Dictionary, Cambridge International Dictionaries, and Oxford English Dictionary.

Online dictionaries. The era of the Internet made online dictionaries available directly from the desktop of a computer, and later from a smartphone. Skinner (Skinner, 2013) noted: “The list of most searched words in Merriam-Webster's online dictionary now includes holistic, pragmatic, caveat, esoteric and bourgeois. Historically, the purpose of lexicography has been to explain unknown words to readers. In addition, modern dictionaries successfully cope with this.” Dictionary.com is an online dictionary, whose domain was first registered on May 14, 1995. The primary content on Dictionary.com is a proprietary dictionary based on Random House Unabridged Dictionary, with editors for the site providing new and updated definitions. (Herman, 2015)

Computational linguistics differs from traditional methods of natural language processing in that in the first case, attention is focused on modeling everything that linguistics studies as a whole, while in the second, the main attention is paid to the division of the process of understanding the language and to the theoretical linguistic correctness and adequacy of the proposed models.

Computational linguistics is closely related to the central problem of artificial intelligence - the elec-tronic representation of knowledge. The main task of computational linguistics is the construction of log-ical-linguistic models and their corresponding algorithms and programs.

The solution to the problem of lexical disambiguation (WSD) and the development of lexicography benefit each other: WSD provides an empirical grouping of meanings and statistically significant indicators of context for new or existing meanings. In addition, WSD allows you to create a semantic network based on machine-readable dictionaries. On the other hand, lexicography provides a larger and better set of senses and a collection of annotations to the meanings of words, which can benefit WSD (Navigli, 2009: 1-69).

Thanks to their functionality, electronic dictionaries possess some advantages over their paper coun-terparts.

Multifunctionality. It is a variety of additional functions that simplify the use of the dictionary. For example, you can specify parts of speech, origin, as well as word formation, taboo vocabulary.

The use of multimedia. It means voicing of heading words, the introduction of illustrative material with photographs, animation, video clips, as well as the use of various graphic means.

Relevance and dynamism. It is the possibility of constant updating of information, as well as the removal of outdated data. This is one of the important advantages over "paper" dictionaries, since they inevitably become outdated at the time of their release.

A large vocabulary base. Most electronic dictionaries' term base exceeds that of paper dictionaries and provides a convenient access to the information.

Variability in use. It is the ability to use dictionaries in local and global networks, namely, the use of offline and online versions.

Universality. As a rule, programs allow you to work with several languages and directions of translation at once. It is possible to use any of the languages included in the dictionary as an input.

Convenient search. It is the ability to use an effective search system (full-text search, simultaneous search in several dictionaries, high search speed). There is also no need to remember the word exactly; the program itself will suggest options for the first letters. Electronic dictionaries use a variety of linguistic technologies to access content, such as morphological and syntactic analysis, full-text search, speech recognition and synthesis.

Tasks of computer lexicography

Scientists and programmers face many problems when converting paper dictionaries into machine- readable dictionaries.

Parsing. This task consists in extracting information from a natural language dictionary and presenting it as a tree structure with nodes that consist of attributes. Each attribute has its own value (Byrd et al, 1987: 219-240).

Creating a lexical database. When creating a lexical database that stores the information of the formed dictionary, it is necessary to provide fast, flexible and convenient access. Besides, the records that will be stored here must have a structured form. The query language for this database should provide convenient addition, change and deletion of information (Byrd et al, 1987: 219-240).

Lexical analysis of electronic dictionaries. This task is to perform an analysis of the contents of the dictionary (lexical analysis of a word, search for synonyms, and search for internal links) (Byrd et al, 1987: 219-240).

Estimation of the number of multi-valued- single-valued words. Most words in dictionaries are unambiguous, but there are words with multiple meanings. The task is to count the number of word meanings in the electronic dictionary and the number of polysemantic words (Amsler, 1980).

Automatic extraction of hyponym-hypernym relationships. The task is to find all hyponym-hypernym pairs for a given dictionary. Since words are ambiguous, any word can have several hypernyms, so if you take all hyponym-hypernym pairs in the dictionary as a group, then by Amsler's definition (Amsler, 1980) it will be a “tangled hierarchy”.

Displaying values when extracting from several dictionaries at once. The task is to compare the meanings of a word given in one dictionary with the values of the same word in another dictionary (analo-gous to the ontology-mapping task). Solving the problem of displaying values is necessary in order to com-bine data from different dictionaries. For example, this would allow grammatical information from the Longman Dictionary of Contemporary English to be correlated with a definition from Webster's Seventh Collegiate Dictionary (Byrd et al, 1987: 219-240).

Defining a value within a single dictionary. The relation of synonymy (hypernymy, etc.) is the relation between meanings, not words. Therefore, the task is to choose the meaning of a polysemantic word within the same dictionary in order to indicate a synonymous meaning. This task is closely related to the previous one, since when comparing the values of several dictionaries, it is necessary to automatically select the corresponding value from the list of all values listed in the dictionary entry (Byrd et al, 1987: 219-240).

Extracting information using a set of monolingual and translation dictionaries. The idea is to take information from a monolingual dictionary for a given language and using a translation dictionary as a transmission device transfer the information of the monolingual dictionary to the second language (Byrd et al, 1987: 219-240).

Principles of a dictionary compiling

In order to set in any of the terms into a dictionary a definite principle of compiling the page occurs. For example, let us consider such term as `bug'. The page in the dictionary looks as follows (Fig. 1, 2. 3).

Further, we shall see macro- and micro-structures of dictionary where macro-structure represents a gen-eral structure of the dictionary, that is content and connecting parts and microstructure is a format of the dictionary entries and the parameters of their filling in.

The main module of a bilingual dictionary is a dictionary entry consisting of lemma, the zone of the phonetic information, the zone of grammatical information, the zone of equivalent and the reference zone.

In morphology and lexicography, a lemma (plural lemmas or lemmata) is the canonical form (Zgusta, 2006), dictionary form, or citation form of a set of word forms. In English, for example, break, breaks, broke, broken and breaking are forms of the same lexeme, with break as the lemma by which they are indexed. Lexeme, in this context, refers to the set of all the inflected or alternating forms in the paradigm of a single word, and lemma refers to the particular form that is chosen by convention to represent the lexeme. Lemmas have special significance in highly inflected languages such as Arabic, Turkish and Ukrainian. The process of determining the lemma for a given lexeme is called lemmatisation. The lemma can be viewed as the chief of the principal parts, although lemmatisation is at least partly arbitrary.

From Wikipedia, the word lexeme in computer science is defined differently than lexeme in linguis-tics. A lexeme in computer science roughly corresponds to a word in linguistics (not to be confused with a word in computer architecture), although in some cases it may be more similar to a morpheme. In some natural languages (for example, in English), the linguistic lexeme is similar to the lexeme in computer science, but this is generally not true (for example, in Chinese, it is highly non-trivial to find word boundaries due to the lack of word separators).

Corpus lexicography is a separate area and a new direction of contemporary lexicography. Corpus lexi-cography is the field of language learning based on text or acoustic corpora with constant use of a computer in certain phases of data storage, retrieval and analysis. Thus, the most popular series of dictionaries is based on the well-known language corpora The Bank of English, lexicographical works of the series Oxford University Press are based on British National Corpus, the dictionaries of the series Longman are based on Longman Mini Concordances.

Another area of lexicography is cybernetic lexicography. Cyber lexicography is the use of the Inter-net to create electronic dictionaries. An electronic dictionary for lexicography is the object of research.

Practical lexicography can be defined as the process of compiling dictionaries of various types on the basis of theoretical developments.

Stages of dictionary development

Fig. 1. A definite principle of compiling the dictionary page (step 1)

Fig. 2. A definite principle of compiling the dictionary page (steps 2, 3)

9. L. Zgusta, F.M. Dolezal, (Ed.). Lexicography then and now, 2006.

REFERENCES