Developing Students’ Skills of Identifying Reliable Scientific Sources: The Role of a Librarian

Размещено на http://www.allbest.ru/

Developing Students' Skills of Identifying Reliable Scientific Sources: The Role of a Librarian

Valeria A. Vasilyeva¹ , Anatoly N. Shilov²



ABSTRACT

Purpose. The purpose of this paper is to provide modes of teaching Russian undergraduate students fundamental critical analysis of scientific papers and ways of identifying misinformation and disinformation in “Academic Writing” discipline.

Design/methodology/approach. “Academic Writing” discipline was taught to first-year undergraduate students by scientific library experts. At a starting period students were asked to complete a questionnaire and perform a test, and then write an essay to identify how students were able to assess Web scientific literature and verify their “tolerance” to inaccurate and fake scientific information. Then contact hours on theory and practice of identifying and assessing information were introduced into the curriculum. At the end of the semester lists of works cited in original essays and final assays were compared.

Findings. The findings reveal that entirely all first year undergraduate students lack the ability to differentiate between qualitative certain scientific information and misinformation and disinformation, and they willingly borrow anything from the Web. However, students acquired necessary skills to assess information critically using socio-technical systems infrastructures. This was demonstrated with list of literature analysis in students' essays and peer-checking.

originality. It is headline news and social networks when it is necessary to distinguish from authentic and inaccurate information. However, skills of critical assessment of scientific publications are of vital importance at a starting period of studies. In publications of that sort falsifications, fabrications and other fake discoveries can lead to reputational and financial risks, and then trigger unethical demeanour of future researchers. It is qualified librarians being experts in electronic resources and data based use who are able to get students to “digital hygiene”.

Practical implications. The results of the paper can be used for teaching in university curriculum. Embedding information literacy and academic writing as well as involving information resources into students' curriculum are aimed at the ability to identify misinformation and disinformation. Librarian-developed “Academic writing” discipline component might be introduced into curriculum as a self-contained short course or a composing element of any students' academic discipline. All this mitigate the risks of applying low-equality, unverified and openly fake information in educational and scientific works.

Keywords: misinformation, disinformation, library expert, information literacy, academic writing, science information assessment, teaching critical thinking, digital hygiene



Развитие у учащихся навыков выявления достоверных научных источников: роль преподавателя-библиотекаря

Васильева В. А.^1, *, Шилов А. Н.²

РЕФЕРАТ misinformation disinformation library expert

Цель. Описать методы преподавания основ критического анализа научных документов и определения фейковой информации студентам бакалавриата в рамках дисциплины «Академическое письмо».

Методы. Дисциплину «Академическое письмо» готовили и преподавали студентам 1 курса специалисты научной библиотеки. В начале курса проводился небольшой опрос студентов и тест, а также было дано задание написать эссе, чтобы определить интересующую преподавателя ситуацию -- умение студентов оценивать научную литературу в сети Интернет и уровень их «толерантности» к непроверенной научной информации. Затем в программу дисциплины были введены специальные часы, посвященные теории и практике определения и оценки информации и данных. По окончании семестра были подведены результаты обучения -- проведен анализ списков использованных источников в первоначальных эссе и в эссе, написанных после изучения дисциплины.

Выводы. Практически все студенты бакалавриата на 1 курсе не могут отличить качественную, проверенную научную информацию от фейковой и легко воспринимают все, что «написано в интернете». Однако после серии лекций и практических занятий с применением инструментов анализа контента информационных платформ у студентов появились навыки критической оценки информации, что показала оценка списков литературы в эссе и взаимопроверка работ.

Оригинальность/ценность. О необходимости отличать качественную, проверенную информацию от дезинформации говорят в основном применительно к новостным лентам и социальным сетям. Однако в высшей школе с первого курса также важно дать навыки критической оценки научных публикаций, где фальсификации, фабрикации и иные фейковые открытия могут повлечь за собой репутационные и финансовые риски, а также спровоцировать неэтичное поведение будущих исследователей. Именно библиотечные специалисты, работающие с электронными ресурсами и базами данных, способны приучить студентов к «цифровой гигиене».

Практические последствия. Материалы данного исследования могут быть использованы в программах обучения университетов и колледжей. Дисциплины, направленные на обучение молодых людей информационной грамотности, академическому письму, работе с информационными ресурсами -- все они сегодня должны содержать темы навыков распознавания фейковой информации. Разработанная библиотекарями часть модуля «Академическое письмо» может быть внедрена в обучение и как самостоятельный курс, и как часть любой дисциплины, связанной с научной работой студентов. Это значительно снизит риск использования обучающимися некачественной, непроверенной и откровенно фейковой информации в учебных и научных работах.

Ключевые слова: фейковая информация, дезинформация, информационная грамотность, академическое письмо, оценка научной информации, обучение критическому мышлению, цифровая гигиена



Introduction

The spread of the word “fake” over the Web has become ingrained in our everyday vocabulary. This term expresses both «false, often sensational, information disseminated under the guise of news reporting» [36] and Web publishing with no distinctive fake features. Detecting fakes is a process associated with informational social propriety. It also serves for securing fakes from being spread on a large scale. Fake aggression is pronounced in two ways. One is a social aspect when individual and public opinions are involved. Another is an extremistic one associated with incitement to hatred and hostility that might later lead to fanning the unrest [41]. Considering the amount of disinformation and the speed it is spread by media we might witness harmful political, financial and social repercussions [43].

Mass media fake content is of attention of many. Scientific library experts are also of great concern in this matter as it is a library that manipulates with scientific information in a capacity of a database explorer for higher institutions. However, experts, examining fake news, are indicative of considerable problems of verifying information as fake news may be false all way through or decontextualized which makes it false in part. Apart from this, there is news that angles real events. To challenge with that kind of information is obviously problematic. Under the circumstances, anti-faking in the Internet is creating a quandary situation. It is only possible when there are more specific media differentiating fake news and legitimate news, particular regulatory acts disseminating fake information, and continuous promoting and educative awareness [41]. The most difficult here is to detect scale and limits of scientific misinformation and disinformation. They consider “scientific” papers as fakes that are compiled from other sources or published with a great number of null or nonentitive references. In this connection, two terms have been introduced: “fake scientist” and “fake dissertation”. A term “fake scientist” is a dissertator presenting falsified or fake data in their research which in no point of fact has ever been done or performed. A term “fake dissertation” is a compilation of text fragments borrowed from another person's works or an obvious supplantation. For example, in someone else's researches they retain all qualitative indicators or quantitative implications, and in their research “meat” transforms into “milk”, the name of a geographical point is substituted for a different one, “managers” becomes “gynaecologists”, etc. [1; 31]. Open scientific misinformation and disinformation appear in nonpeer-reviewed, junk or predatory publications. In scientific discourse, for predacious publisher or predacious journal researchers they have discovered synonymies “false”, “pseudo”, “discredited” [19]. Other researchers [7] identified three broad types of fake information, namely, fabrications, broadscale hoaxes and humorous fakes. The term “misinformation” is also used, which refers to information that might be treated as honest mistakes [48].

Whatever term for fake publications is used it always leads to harmful effects. Whether such kind of information is spread through social network or news feeds or publications in journals it brings uniform hazard for society. Many researchers [2; 23; 24; 34; 44] paid a great deal of their attention to harmfulness of fake information for younger generation. However, scientific disinformation is to be under close research as a fake in science may do much more damage to society if compared to public grievance it might have from a fake about celebrities. This has become particularly clear in widespread danger of COVID-19 when numerous so-called researches might have had a great negative impact on public life [51].

A well-known story tells that three scientists purposely, for an experiment, created fake articles on social issues distributing them in journals. As a result, minimum seven out of total twenty articles were published, while their contents were not to be able to withstand even a moment's scrutiny. With such an experiment researcher indicated the possibility of manipulations in scientific world. Manipulating may carve out a place in international scientific environment [27]. The publications generated by artificial intellect can also be treated as fakes, e. g. GPT-3 autoregressive language. Researchers in France and Russia detected an increase of publications of an incomprehensible and malformed contents and texts of empty meaning in Microprocessors and Microsystems, Elsevier publisher [5]. Along meaningless content such publications have erroneous references on unreliable and nonexistent sources, which can harm world science.

Detecting fake scientific information is not an easy thing even for professionals as scientific publications are mostly published in scientific journals and by reputable publishers. In 2020, Royal Chemical Society retracted nearly 70 published articles from their journals having detected systematic reproduction of falsified researchers. That activity got the name “paper mill” [49].

Occasionally, information is produced in a form of images, diagrams, tables or figures, so visual information is not easy to be recognized. Researchers in Great Britain proved that people are bad at identifying whether the image is true or undergone changes, using graphics editor, for example [32]. J. Schwartz [39] justifies necessity of introducing a separate paragraph on visual literacy into information literacy. This shows that fake takes more sophisticated forms. In some cases, it may not be recognized. Distinguishing between fake and qualitative information is not easy even for a professional with no specialized knowledge. It is notedly for Humanities. The last decades witnessed involution with pseudoscientific researches and publications. They are hardly distinguishable from quality publications, and in-depth study and rigorous evaluation of contents, structure, author's individual contribution and evidence of theses can help here. On the other hand, citation with no comments, not well-established complex scientific definitions, loose interpreting of terms or ambitious self-citation can also be indicative of low-quality scientific information [45]. No doubt, a younger generation student lacks relevant skills for objective reasons, which is illustrative of numerous researches all the world around.

In 2016 a group of Stanford University experts [42], conducted a challenging research aimed at estimating fundamental information competences with 1,000 young people all over the country. Those involved were secondary school, senior high school and college students. The participants were invited to use Web-resources of different levels -- Fa- cebook or Twitter news, online articles, Web-sites images, social networks posts, etc. depending on the type of a school or college. The obtained results were variegated and engaging, but general conclusion was rather disappointing -- “digital natives” are very easily misled when it comes to evaluating or estimating information [42, p. 4].

Centres of academic writing or their counterparts are of great assistance to researchers and other scientific publishers all over the world. Those Centres secure assistance in evaluating quality of scientific information for citing. However, there are not many centres of this type in Russia, while there are a big number of educational institutions. In 2017, there were 14 centers but only 7 were really functioning [4]. The thing is even worth with “Academic Writing” discipline for students. While many professionals unanimously advocate for introducing that discipline into curriculum at a starting period of learning, better in school years, it has not become a part of curriculum at universities nowadays.

Once again, we have to make “Academic Writing” a mandatory discipline that teaches critical analysis of information, helps identify disinformation and leads to recognizing true materials borrowed from the Web.

Students are supposed to be taught to work with scientific information. We are in full alliance with those who think that it is a hard job for students to puzzle out the Web materials all by themselves [14; 30]. By no means all school curricula include information literacy disciplines. It means that in case there are no disciplines teaching university students how to differentiate between authentic and fake information, budding scientist and would-be pundit, objective and falsified research, then graduates will not be able to work with scientific information.

This study investigates teaching methods of critical evaluation of scientific information skills for first-year undergraduate North-West Institute of Management, branch RANEPA in their “Academic Writing” discipline. This discipline had been developed and taught by scientific library experts, who had been dealing with scientific information for years. Survey findings and tests at a starting period proved the idea of introducing contact hours for information analysis. The findings demonstrated low cognitive ability with first- year students to assess scientific materials properly. At the end of studies we estimated an acquired theoretical knowledge level and practical skills of students reasoning from produced essays and coursemates' peer-review.

Literature Review

Much has been written about information literacy. Researchers outline specialized disciplines, courses and seminars on information literacy teaching methods both in colleges and higher educational institutions [40; 47; 50] and public libraries [29]. It is mostly librarians who organize those courses and seminars [46] since information resources in their traditional, paper and electronic format are accumulated in libraries. Besides, academic librarians work directly with information consumers -- lecturers, students or participants in the course. They know problems related to their consumers' information literacy better than most [10].

A great number of information literacy courses promote critical thinking skills for qualitatively relevant information selection [33; 38]. V Giri and M. Paily [15] examined the effects of applying the model of Toulmin's argumentation to enhance critical thinking upper-formers skills in secondary schools. They demonstrated this model to be more efficient in promoting critical thinking skills compared to a traditional approach.

However, it is not the entire picture of the matter concerned. In a broader sense, critical thinking is a keystone of civic engagement and inclusiveness [35], which makes it the main competence of the 21^st century [14].

In this connection, it is becoming particularly important to teach students how to evaluate information from their first steps in a higher school. Young people find themselves in continuous communication when the primary means of communication for them are social networks and news feeds. This is the reason why researches on fake news and publications mainly concentrate on inaccurate or deceptive information being spread through multiple social media platforms and news feeds. They demonstrate methods in the battle against fake news via media literacy course [17; 29]. Recent research conducted by some researchers [34] reveals urgency of fake news in 2018. It also tells about wide spread of media literacy course in school curriculum.

The Role of Librarians in Promoting Students' Critical Literacies

Students find it difficult to evaluate adequacy and authenticity of educational and scientific information borrowed from the Internet. At that time they are not alive to the importance of the matter [26]. To identify students' true cognitive skills of literature selection some specific strategy and arrangements (tests, surveys, experiments, etc.) are required [8; 35]. Once the real matter of things becomes clear students are being taught skills of how to deal with scientific information. Along with traditional platforms of seminars and learning courses there might be unconventional forms of journalistic clubs [9] or activities on evaluation of articles containing deliberate mistakes [11].

It is library professionals who are busy with teaching media literacies skills at colleges and universities. Their role in a capacity of information professionals is on an incremental increase [13; 18; 33; 37]. Walsh [48] states that when teaching students how to evaluate information reliability librarians are supposed to teach how to identify disinformation. Disinformation and fake news are on a larger scale, thus it becomes difficult for librarians to enable students find ways of identifying reliable sources of information [28]. However, to prepare students to critically absorb information teaching librarians embrace aspects informational, media and digital literacies, in other words, “digital hygiene”. At purpose seminars librarians introduce active components to see how reasonably students can detect fake news to make it clear if there are any vacancies in students' knowledge and assist them in getting required competence [3].

The literature emphasizes the urgent necessity of promoting critical thinking with students at colleges and universities nowadays, when social, scientific and personal communication is moving online. Students make at-risk group when they become consumers of fake information and disinformation. Librarian professionals fill in the blanks in pedagogics teaching critical evaluation of science literature for first-year undergraduate students in their “Academic Writing” discipline.

Research and Teaching Methodologies

Students are taught “Academic Writing” discipline in the first semester in their first year,

i. e. at a starting period of education. “Academic Writing” discipline is a credit bearing course of three course credits. That equals 108 academic hours, which split as follows: 16 hours for lectures, 32 academic hours for practical studies, and the remaining 60 academic hours go to self-study. The purpose of the discipline is to develop writing skills and to teach an academic text strategy, rules of citation and referencing for first- year undergraduate students. The discipline lays the groundwork for students' further disciplines where writing academic essays, scientific articles or any other kind of written tasks are required. In our particular case, there were totally 146 students for two academic years. There were young adults of 17-18 years old, who entered the Academy immediately straight after leaving secondary school. Before 2020 the discipline “Academic writing” had not been a part of academic curriculum, that it had not been taught to students. Since 2020 we have estimated effectiveness and usefulness of teaching thus discipline within two years. The results presented illustrate the scenario.

This made it reasonable to conduct a research to find a staging point to proceed further. It was dramatically important to realize whether students felt the difference between fake and verified information and where they were familiar with testing and checking it. The research was especially important to carry out because, as we have noted earlier, students are aware of their ability to “work” with information since their early years in the Internet and they see no problem in searching data. To motivate students for learning it becomes essential for professionals to realize where precisely students lack cognitive skills to perform relevant job sensibly.

The research was split into three stages, namely:

(1) Quick recitation

(2) Test on how to evaluate a scientific text

(3) Analysing essays written by students

FIRST STAGE. The recitation included the following questions:

• Where do you receive information for your scientific paper?

• Can you distinguish between reliable and unverified science information?

• What makes scientific information different to any other (news, journalism, work of fiction)?

• What is quality indicator of science information for you?

As to the first question, totally 100% students responded that they borrow information from the Net. At the same time, they displayed full ignorance of electronic platforms providing access to scientific publications and data -- Dimensions, Microsoft Academic, Scopus, Web of Science, etc. Not so many of them were familiar with Google Scholar and Russian Science Citation Index (RSCI).

The students also failed to differentiate authentic scientific information from unverified one. They were not able to call any checker. There were occasional responses when students treated the information as authentic in case they received it from their lecturer or teacher.

When asked a question what makes scientific information different from any other they failed to answer. Among the features that students contrasted between valid scientific information and unreliable information were scientese and academic voice, statistics and research methods used in scientific papers. However, reviewing process, when the article was reviewed and was considered scientific valid, remained vague for the students.

As to quality indicator for science information half of the students specified author's reputation, scientific style of paper, true references. However, how to identify the reputation of an author students were not able to tell. Moreover, their knowledge of source reference quality was limited to Wikipedia.

Thus, the recitation revealed broad vacancies in students' knowledge on how to estimate authenticity and quality of scientific information.

SECOND STAGE. Students were asked to perform a test containing three reference links to articles of free access in the Net. The first reference made a link to a public article borrowed from Elsevier Publisher https://www.sciencedirect.com/science/article/ pii/S0148296319304564 (Fig. 1).

The second reference linked to an article borrowed from Social Science Research Network (SSRN) Repository, where authors are in a position to download their published works and preprints at their own discretion, which are censored afterwards -- https:// papers.ssrn.com/sol3/papers.cfm?abstract_id=3226476 (Fig. 2). The third article had a reference link to an article under a fictious title in “The Atlantic” multiplatform publishers, in their “Science” sector -- https://www.theatlantic.com/science/archive/2016/08/ sexism-in-the-stars/496037/ article-comments. The article was produced by a group of scientists making a social experiment spreading fake articles in different journals (Fig. 3).

Students were asked to distinguish between reliable and “suspicious” scientific papers. Students were not unanimous responding to that task. However, vast majority of students unfailingly identified a scientific paper in Elsevier Publisher's journal as a credible, verified and rigorously science-based one. Students also split over the two remaining papers, when half of them identified a SSRN Repository public article as a reliable paper while another half estimated The Atlantic popular science journal article as an original one. It is of particular note that students' choice in favour of reliable paper was not a result of clear and rational basis as they were not able to produce evidence of their decision. It was a matter of “intuition”, of sensation rather than a reasoned choice.

THIRD STAGE. Students were asked to write a short academic essay on any topic of their interests. They were not posed any particular requirements or regulations or specific directions but only one -- their essays should contain bibliography.

Thus, at the initial seminars it became clear which particular aspects were to be of the most concern.

Students' essays contained very few valid references making a link to Web pages, Wikipedia specifically. In most cases the references did not reveal links to the source of information (a journal). There were links to URL but not the author's name. Apart from this, it became clear that students, when choosing such-like materials for citing in their essays, were easily bought into demonstratable features of fake information, such as impressive set expressions, nameless source materials (“British scientists”), manipulative semantics [43], etc.



Fig. 3. Article from the Atlantic

Initially, “Academic Writing” discipline did not contain any components dealing with selection and analysis of scientific information, although principles and skills of citation made the subject of the lecture. Having performed the survey, we made some revisions in the curriculum for “Academic Writing” discipline with eight academic hours for theoretical and practical aspects of evaluation of scientific information.

Thematic discipline comprising eight academic hours was split into three components as follows:

• evaluation criteria by which reach and authenticity of information are ascertained (4 academic hours)

• reviewing scientific information as a quality attribute of an article (2 academic hours)

• ethics of source citation (2 academic hours)

Teaching Methods

Criteria for information evaluation

In their theoretical component (4 academic hours) students were explained the main criteria of reach and value of papers, namely: publisher's diagnosis; author's assessment; structure and content of a paper.

Such kind of approach for scientific information reliability assessment was introduced by P. Ky [25] where he made an assessment from four sides: authors and researchers' credentials, science-based feedback, peer-reviewing, and evaluation of scientific method.

As to how to diagnose a publisher and assess an author students were supposed to conduct an analysis using bibliometric data basis -- Russian Science Citation Index, Scopus, Web of Science, alongside with public repository platforms and science social networks. Students were likely to develop the following skills:

1. Ability to search information to make sure the journal pertains to some publisher or some other institution, estimate time of origin of the journal, remit of the journal, editorial board composition, identify the rank of the journal in the world of science by means of research indicators;

2. Ability to identify author's affiliation to some institution, assess their publishing portfolio, academic field of interest, co-authorship;

3. Ability to assess a paper in terms of its structural composition, arguments, references and bibliography.

We made students clear that a risk of getting scientific disinformation becomes greater in case of placing it on uncertain sites or publishing it in unidentified and little- known journals. Special attention of students was drawn to verifying author's credentials, namely: wherever researcher's works were published, what their working area or affiliation is, who their co-authors are, if their contacts and affiliation are valid.

An infrastructure by means of which students were expected to identify and verify author or publisher's credentials was displayed through Web of Science and Scopus lists of journals indexed, Publons platform, SSNR and ResearchGate science-based social networks, and optional Dimensions and Microsoft Academic platforms.

In case of an obvious fake publication (above-mentioned part of the test with The Atlantic article) we demonstrated that there is no for-real author with such a name, no database contains any relevant information about them, and no possibility to identify their affiliation. All this is expected to get attention of those who use science literature.

In some cases, structural composition and content of scientific paper can assist a reader to differentiate a reliable scientific article from a fake one. Majority of journals follow a structured format for an article written, e. g. IMRAD (Introduction, Methods, Results and Discussion), that makes every aspect of a research clearer. Having examined IMRAD formatting, we emphasized that a scientific paper is to contain three essential elements as follows:

• theoretical background, i. e. literature review of the research of concern,

• a clear in-depth description of research methods,

• comprehensive and coherent description of findings.

Why do scientific articles contain these mandatory elements? This is for reason if a research and its article are not fake, any curious scientist might have a possibility to duplicate the research in whole or in part. There is also a possibility to duplicate only a part of the research, for example, use described methods with the aim of gaining their own alternative findings for a particular situation. Or, the findings in both researches are similar but the methods used were new. Whatever the case is science is progressing: researchers make arrangements for scholarly disputes and reveal new findings.

When a paper contains research methods obscurely described, findings not stated, literature review ignored, references are very few, worth a thought if it is true but a fake. Students were demonstrated every element in their case studies asking them to come up with validated and well-thought assessment.

Students were also warned of the importance of obvious note “Retraction”. Risk of using retracted publications is boosting, especially when searching for literature in bibliographic database. A case in point relates to articles retracted by Royal Chemical Society in 2020. Students were shown a search result based on Scopus scientometric base citation detecting a fake article and notifying it retracted [12]. (Fig. 4).

Inexperience in information analysis stymies a reader to detect a fake in the initial publication. Students were alerted to pay particular attention to the warning “erratum” or “retraction” to make them realise whether the paper is authentic and reliable or fake.

Reviewing scientific information

Reviewing scientific information became the second thematic component of a revised curriculum of “Academic Writing” discipline with 2 academic hours. We were to intent on making students understand that reviewing is the main criterion of distinguishing between reliable and fake information [26]. Leading journals and reputable publishers maintain their own staff of reviewers who bear their responsibility for the information presented with them. Availability or deficiency of scientifically acceptable reviewing is also likely to be detected while assessing scientific information borrowed from the Net. Analysing reviewing of a publisher students' attention was drawn to Publon platform where they could find reviewers' credentials, journals' reviewing record, list of confirmed reviewers on a particular discipline, region and concrete journal.

Students were extremely impressed with clear and easy to understand reference of disinformation: unverified information about causes of obesity or coronary heart disease when “saturated fat is the major dietary villain” [21], ozone layer depletion harmed by deo sprays [20] , seventy articles of Royal Chemical Society containing disinformation about chemicals and healthcare products for various diseases [6].

Ethics of source citation (2 academic hours)

Special students' attention was paid to ethics of dealing with someone else's texts, getting skills of accurate citation, detecting wrong, null or faulty references in science- based texts that shall hamper complicate scientific communication. Some warning was made to avoid referencing on Wikipedia, personal blogs or social networks in academic papers. Students were explained that using fake references from someone else's unverified texts shall spread fake information even where the author's text is valid and reasonable.

As a practical task students were asked to identify the primary source of information following references from the list of bibliography for a published quasiscientific article in a junk journal of poor quality papers. When completing the task students were astonished that following the references they were not able to detect indicated sources of information but found non-existent Web pages, and then “phantom” publications. They felt screwed, which was the best way to demonstrate how important it is to acquire proper citation skills.

Learning and Training Outcomes

Impact of teaching methods and principles for “Academic Writing” discipline was clarified upon examination results. First, students were to be ready for their performance appraisal composed of two parts:

• a written essay when students were offered either to stick to their original topic or choose a new topic for them,

• educational “double blind reviewing” role play where essays performed were randomly split among the students involved to check their coursemates' essays.

Then, on condition the two parts were performed successfully students were to take an exam where they orally responded to the examiners' questions.

The essays students presented for their exam were dramatically different to those they displayed initially at a training start. Considering the purpose of the paper is methods of teaching critical analysis of scientific literature via the Net validity of students' essays in terms of academic writing, style, thoroughness of topic's elucidation remains beyond the scope of the research. The essays were scrutinized for using fake publications, unconvincing and unverified information, inaccurate and inadequate references in bibliography.

In total, 132 essays were presented and compared with 141 essays presented ad initium. Prior to a training start 42 essays out of 141 presented contained no references to the sources used, thus they were not considered for comparison. For a similar reason, 12 essays out of total 132 at the end of teaching period were withdrawn but it is worth mentioning that the essays with no references to the sources used were 10% fewer.

Remaining 120 essays were examined for the efficient use of applied methods of critical analysis of publications. At this stage the following results were obtained:

(1) references to popular journals with unverified information via the Web -- 13 essays;

(2) references to licence information from corporate subscription resources (recommended) -- 98 essays;

(3) references to traditional paper publications (books) -- 27 essays;

(4) reference to legislative and regulatory documentation -- 12 essays;

(5) references to fiction -- 6 essays;

(6) references to credible sources with verified information -- 46 essays.

A clarification should be made: a number of essays contained more than one types of references mentioned above.

Then the references produced were summarized and classified as verified, nominally verified and fake. The publications borrowed from licence materials (subscription resources), references to “golden open access” materials, regulatory documents and textbooks were treated as verified. References to fiction in its paper or electronic form were considered nominally verified. References to Wikipedia, advertising literature, blogs, popular tabloids and with-no-name Web pages were classified as fake.

Results obtained and results received at a training start were compared (Table). Taking into account that the number of essays under examination differed all data were summarized and translated into quality approved approach (Fig. 5 and Fig. 6) Results obtained demonstrated positive fallout from relevant teaching sessions., However, we have not managed to uproot students' habit to refer to unknown pages in the Internet. 10% of works contained that kind of references. We assume the reason for that disappointing result was an unexpected transition to the hybrid education at the peak of second-wave corona virus pandemic when one of the most efficient ways of skills consolidation (public discourse or dialogue) became no longer possible. Another possible reason might be for those students who missed some sessions.

At the second stage of the task students were offered to take part in an educational game called “double blind peer reviewing”. Students were asked to evaluate someone else's essay by using 3D discipline method introduced by B. Green [16]. This method

Table

Reference distribution per verified, nominally verified and fake in bibliography sources

Essays, total	Starting Point -- 99	Completion -- 120
References, total	95	164
References to 1st essay	1,0	1,4
References verified	30	143
References nominally verified	15	6
References fake	50	11

J Verified

] Nominally verified ] Fake

Fig. 5. Distribution of sources of information prior to a training start

Fig. 6. Distribution of sources of information upon discipline completion

] Verified ] Nominally verified ] Fake

implicates three-dimensional text evaluation -- operational, critical and cultural. Students were supposed to evaluate each dimension according to a points system (with maximum points 100), then they produced a narrative summary following no particular form, i. e. using own words. In a capacity of a reviewer students felt much more confident and could fairly identify inaccuracies. They gave their coursemates' lists of literature more meticulous attention compared with their focusing on the matter in their own essays. Such kind of a play inspired great students' interest proving suitable and productive way of refreshing or reiterating information evaluation. Some summarizing narratives made it clear that students had soundly digested the discipline content. In one of the summaries a student says:

“The arguments produced seem unconvincing easy assumptions and generalities. The author refers to general fiction rather than science-based articles, news or research results or findings. A number of unfounded claims are produced. The author blames Russia in autocracy or Bill Gates in conspiracy to dispeople the planet. That sort of broad statements should be underpinned by science-based articles or research evidence...”

Findings and Further Research

Librarians -- both in Russia and many other countries -- contribute greatly into promoting students' information literacy. It is worth noting that disciplines aimed at boosting critical thinking require undivided attention and continuous refreshment and perfecting. What is of primary current concern is detecting relevant information alongside with lowering the risk of using fake publications in all their varieties.

One major detected problem in terms of initial assessment of information with students is the fact that students ignore clear “hooks” used by those who spread fake information, i. e. particular headlines, emotionally charged language to attract special attention, conspiracy theories and many more. Such type of information normally top an inquiry answer in search engines and presented in clear simple language, which for an inexperienced day-old school-leaver might seem authentic.

Problems arising from low cognitive students' skills of detecting disinformation and fake information are due to the fact that there are no widespread academic literacy disciplines in academic curriculum in Russian colleges and universities. Academic writing, critical thinking, information evaluation, policing content on communication platforms disciplines are by no means taught in Russian higher institutions. Unfortunately, it was only twelve years ago when foreign practices of teaching academic writing and critical thinking reached Russia being introduced in colleges and higher educational establishments through Academic Writing Centres recently setup. It is only nowadays when they are getting widespread use.

Experience gained proved that eight academic hours were fairly enough to teach students to lower the risk in their battle against fake science-based information for their self-study within “Academic Writing” discipline framework. On separate occasions, we taught graduate students how to deal with academic texts. Those students had not had “Academic Writing” discipline in their curriculum, nor had they had any academic hours for learning information verification skills. In most cases students distributed sources of information into verified, nominally verified and fake as poorly as our current students used to do at their initial steps of learning the discipline. It becomes obvious that it is only in their first year, at the start of their tuition, education students have a possibility to acquire the required writing skills, most probably with the help of library experts only. They will have no further opportunities for getting these skills. In their second and third years of undergraduate programmes students are focused on their major, vocational and qualification subjects. They have no possibilities of getting academic writing skills or, what is even more important, improving their information verification skills since their academic curriculum has not been timetabled for that purpose. In fact, the situation remains in place.

The fact that while peer-checking and assessing their coursemates' essays students displayed better understanding of academic writing principles, source use etiquette, and scientific information evaluation rather than in their own produced essays can be interpreted differently. On the one hand, students acquired fairly sophisticated skills in their ability to evaluate and assess academic texts. On the other hand, students lacked practices of producing their own texts within limited academic hours for “Academic Writing” discipline. We have full understanding of the fact that the discipline is to be expanded and enlarged, introducing extra academic hours for identifying information teaching. The mandatory discipline are as follows:

• Research factors and scientometric indicators (in order to be able to get clear information about journal rating, author's reputation and their affiliation to a relevant institution).

• Ethics in borrowings from texts (where and how to make references to science-based information).

• Information databases and platforms to be able to work with information and data, most of which are part of bibliography managers.

• “Predatory” and clone journals (it is them which spread unverified information and are fake in their nature).

• Guidelines of using drawings and illustrations borrowed from other authors' articles

• Principles of retracting scientific publications, etc.

Notwithstanding, whatever continuum of any abilities and skills, critical thinking and information selection skills included, there is no ending headway [22]. Improving and mastering the skills is a life-long process. A university teacher's challenging aim is to provide a framework for students' further progress in their earliest years of education. We will teach young people to think critically assessing information acquired, avoid using fake news or falsified researches, analyse, dispute, challenge and verify, and then students will become scholastic informationally-armed competent professionals.

In the scope of current research there is no answer to the question whether the skills students obtained shall stay with them further. Today they are second-year undergraduate students, and we aim to continue our mutual collaboration with them but in a different format, for example, by arranging specialized seminars or involving them into “writing group” club.



References

1. Abalkina A. (2019), `Scientific falsification'. Troitskiy Variant -- Nauka, 270, 12. [Абалкина А. Научный фальсификат // Троицкий вариант -- наука. 2019. № 270. P 12]. (In Rus).

2. Allcott H. and Gentzkow M. (2017), `Nber Working Paper Series Social Media and Fake News in the 2016 Election'. Journal of Economic Perspectives, 31, (2), 211-236.

3. Auberry K. (2018), `Increasing students' ability to identify fake news through information literacy education and content management systems'. The Reference Librarian, 59, (4), 179-187.

4. Bazanova E. and Korotkina I. B. (2017), `Russian Writing Centers Consortium'. Higher Education in Russia, 211, (4), 50-57. [Базанова Е. М., Короткина И. Б. Российский консорциум центров письма // Высшее образование в России. 2017. № 4 (211). Р. 50-57]. (In Russ.).

5. Cabanac G., Labbe C. and Magazinov A. (2021), `Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals', arXiv:2107.06751, 0-3.

6. Christopher J. (2021), `The raw truth about paper mills'. FEBS Lett, 595, 1751.

7. Conroy N. J., Rubin V. L. and Chen Y (2015), `Automatic deception detection: methods for finding fake news'. Proceedings of the Association for Information Science and Technology, 52, (1), 1-4.

8. Eldermire E. R. B., Fricke S., Alpi K. M., Davies E., Kepsel A. C. and Norton H. F. (2019), `Information seeking and evaluation: A multi-institutional survey of veterinary students'. Journal of the Medical Library Association, 107, (4), 515-526.

9. Esisi M. (2007), Journal clubs'. BMJ, 335, (7623), s138 LP-s139.

10. Faix A. (2014), `Assisting students to identify sources: An investigation'. Library Review, 63, 624-636.

11. Ferenc J., Cervenak F, Bircak E., Jurikova K., Goffova I., Gorilak P, Huraiova B., et al. (2018), `Intentionally flawed manuscripts as means for teaching students to critically evaluate scientific papers'. Biochemistry and Molecular Biology Education, 46, (1), 22-30.

12. Fisher L. (2021), `Erratum: Elevation of USP4 antagonizes oxygen glucose deprivation/reoxygenation-evoked microglia activation and neuroinflammation-mediated neurotoxicity: Via the TRAF6-NF-kB signaling (RSC Advances (2019) 9 (23916-23924) DOI: 10.1039/C9RA03614H)', RSC Advances. Royal Society of Chemistry, 11, (8), 4374.

13. Fong C. J., Kim Y, Davis C. W., Hoang T. and Kim Y W. (2017), `A meta-analysis on critical thinking and community college student achievement'. Thinking Skills and Creativity, Elsevier, 26, (June), 71-83.

14. Gelder T. van. (2005), `Teaching Critical Thinking: Some Lessons From Cognitive Science'. College Teaching, 53, (1), 41-48.

15. Giri V. and Paily M. U. (2020), `Effect of Scientific Argumentation on the Development of Critical Thinking'. Science and Education, 29, (3), 673-690.

16. Green B. (2012), Literacy in 3D: An Integrated Perspective in Theory and Practice, edited by Beaves, C., ACER Press, Camberwell.

17. Han K. and Kingsland E. S. (2020), `Fake or for real? A fake news workshop'. Reference Services Review, 48, (1), 91-112.

18. Ince S., Hoadley C. and Kirschner P A. (2019), `The role of libraries in teaching doctoral students to become information-literate researchers: A review of existing practices and recommendations for the future'. Information and Learning Science, 120, (3-4), 158-172.

19. Inouye K. and Mills D. (2021), `Fear of the academic fake? Journal editorials and the amplification of the `predatory publishing' discourse'. Learned Publishing, 34, (3), 396-406.

20. Karim A. A. (1975), `Unshielding the sun. Human effects'. Environment, 17, (3), 6-14.

21. Keys A. (1995), `Mediterranean diet and public health: personal reflections'. The American Journal of Clinical Nutrition, 61, (6), 1321S-1323S.

22. Korotkina I. (2020), Models of Training in the Academic Writing. Foreign and Domestic Experiences, Yurayt., Moscow. [Короткина И. Б. Модели обучения академическому письму. Зарубежный опыт и отечественная практика : Монография. Москва : Юрайт, 2020.]. (In Rus).

23. Koshkarov N. N. (2018), `Fake News: Creative or Fraudulent?'. Bulletin of Tomsk State Pedagogical University, 191, (2), 14-18. [Кошкарова Н. Н. Фейковые новости: креативное решение или мошенничество? // Вестник Томского государственного педагогического университета. 2018. № 2 (191). С. 14-18. DOI 10.23951/1609-624X-2018-2-14-18]. (In Rus).

24. Kumar S. and Shah N. (2018), `False Information on Web and Social Media: A Survey', 1, (1), available at: http://arxiv.org/abs/1804.08559 (accessed 10 August 2021).

25. Ky P (2012), `Qualifications, Weight of Opinion, Peer Review and Methodology: A Framework for Understanding the Evaluation of Science in Merits Review'. Journal of Environmental Law, 24, (2), 207-251.

26. Leung J. S. C. (2020), `Students' adherences to epistemic understanding in evaluating scientific claims'. Science Education, 104, (2), 164-192.

...