Rethinking online product review usefulness

Authority claims can take 2 forms: (1) a statement about occupation, where an author discloses his professional experience in the field (for example, “As a web designer/dentist/electrician of 20 years I think that …”) and (2) a statement about amateur or user expertise (for example, “I've been using true wireless earbuds for 5 years already and have experience with models of every major brand”). These claims aim to provide authority-based justification for the following subjective expert judgment calls made by the author.

Besides self-made claims of authority relating to expertise with ALL products/services similar to the one reviewed at the moment (i.e., comparisons with and mentions of other competitor products based on both their qualities and the timeline of their utilization), some big online market platforms allow you to specify for how long the reviews product had been in use before the review was written. Usually they bin periods of user experience in “less than a month/a few weeks”, “several months” and “more than a year”. We will call this period reviewer's “level of expertise”. It can be expected that reviewers that held on with writing a review will give a fairer assessment of the product, as they had more time to identify and validify its strengths and weaknesses, even if fair assessment itself was not the initial purpose of reviewing delay (for example, if the reviewer simply forgot about writing a review). In a sense this measurement of author's expertise can be used as a quasi-indicator of review quality level. Since customers want to spend their time only on the most thought out and fair reviews, those reviews whose authors had waited longer before writing them will receive more attention and be seen as more helpful. This relationship is expected to hold even if customers themselves don't filter reviews or judge their helpfulness based on the described “level of expertise” indicator.

When trying to assess how much trust to put in a particular review, customers may also address the features of authors account. For instance, customers can pre-screen if the reviewer has avatar photo or if he is even a registered user. Anonymous users are trusted less, because people associate anonymity with avoidance of responsibility. They often desire to at least know whom to blame in case the advice they were given will have proven to be suboptimal. Lack of non-default avatar might signal that a reviewer is not an ardent enough platform user. People might believe that if the reviewer hasn't put effort in changing his avatar, his review will suffer from the same lack of effort and thus be of a lower quality.

Review informativeness

It is common sense that more informative reviews are more helpful to the customers and presentation of information is no less important than the information itself: it is desirable that reviews only contain relevant, useful information that can be used for the underlying product's analysis and that those reviews have a clear, coherent structure with negative and positive arguments separated in their respective sections and a conclusive statement in the end for easier following of the author's thought process.

When we speak of “review informativeness”, we will also refer to the contents of individual sections of a review. Those sections can be classified as harmful and beneficial. We will start with discussing harmful review choices: emotional tantrums (for example, overly excessive praise or condemnations), issues unrelated to qualities of the underlying product (e.g., issues with the distributor: damaged package, long shipping periods, inconvenient payment methods, etc.), text bloated with unnecessary epithets and other uninformative parts of the review inflate review's word counts and don't really provide any useful information for product's assessment. Reviews become harder to read and consume more of their viewers' time. The less words are used to state and justify a single claim (higher claims density) about the qualities of the product - the better (of course, if time conservation is your primary objective, which we assume it is). On the other hand, the more actually relevant statements about product's qualities an author makes - the more informative and thus more helpful his review becomes.

Beneficial review choices include analytical segments, price/quality comparisons, evaluation of product description truthfulness, concise summarization of user's experience and indicators of measurement.

Analytical segments may include comparisons of usability, convenience of use, ease of repair and procurement of spares, access to maintenance and support services, quality and price of review's underlying product and its competitors as well as forecasts of upcoming models' release dates and qualities. Analysis can also mention complementary and associated products, for example, how they could increase underlying product's efficiency or its valuable output. Presence of any comparative analysis between the underlying product and its competitors, as well as evaluation of effects associated and complementary products will have on it, would provide a lot of support for the purchase of the leading product, relieving customers from blind assessment. We believe, that the contents of a review can be evaluated by locating keywords necessary to convey specific ideas. To indicate the presence of analytical segments, we will look for proxy-keywords and word combinations like “in comparison”, “difference(s) between” Of course, here we mean their Russian equivalents (including synonyms and speech part variations) and their derivatives. Reviews, containing these keywords are expected to have a higher helpfulness rating.

Price/quality comparison segments are easy and straightforward: they describe how well the underlying product performs relative to its price. Keyword indicators for these segments must include word combinations like “price/quality”, “price-quality” or “price relative to/in relation to/to quality”. Customers benefit from these segments, because they offer a base value model to compare competing products of different price categories (e.g., very cheap generics vs big brand name drugs). Reviews, containing these keywords are expected to have a higher helpfulness rating, especially for products of lower price categories (for cheap products).

Segments, comparing manufacturer's claims with product's actual features and performance are useful to give an idea of how well product's frontpage description fits reality. Does the “what you see is what you get” principle apply to product's pictures and description of its properties? Do customers tend to expect something else from what they receive in the end? Segments like this are very useful to the customers, because they question the most objective information about the underlying product, that many would otherwise take for granted. They are expected to contain keywords like “unexpected”, “as promised”, “correct” (referring to descriptive frontpage information), “the description does/doesn't …”, “… than the picture” (when comparing), and others. It follows, that reviews, containing these keywords are expected to have a higher helpfulness rating.

Segments with opinion summary convey the general level of satisfaction the user had with the product. Customers are quick to understand an author's overall sentiment about the product when they see keywords like “satisfactory”, “good”, “bad”, “falls short of”, “fitting”, “underdeveloped”, “high quality”, etc. Whenever there is a clear indication of sentiment, viewer need only spend their time and efforts on analyzing review's claims about the product. Therefore, reviews, containing these keywords are expected to have a higher helpfulness rating.

Segments that feature qualitative measurements of product's characteristics or the product overall are expected to provide a quick and easy way to compare the underlying product with other options on the market. Words like "the most / the least", "very", "too much", "excessively", "enough" and others alike can not only describe a quality or a relation of one aspect to the other, but also provide an insight into the satisfaction levels the review author has had with the product. Reviews, containing these keywords are expected to have a higher helpfulness rating.

We also identified content segments that are universal among all product types, which we can not categorize as either “beneficial” of “harmful”. We, of course, talk about conclusive calls to action. By “calls to action” we mean lexical constructs addressed directly to the reader of a review and written with a purpose to invoke a desire to act or feel in a certain way. For example, if an author writes “I (or his/her acquaintance) recommend(s) [this product]!”, "I guarantee …", "I'm telling you, …" or something alike, he/she is likely to instill some feeling of security in a decision to buy (or not to buy, if the author recommends against purchasing a product) in a reader, because now a review that this reader has just read is backed up by its author's reputation. In other cases, an author might write something like "Take/Try one!", "… You won't regret it!" or, on the contrary "Beware …" or “Don't even think of …” to try and persuade or dissuade people to act in a certain way without putting his/her good name on the line. In either case it will be interesting to explore whether people feel motivated by, displeased by or indifferent to these calls in a review and how it affects their feedback on them.

Another universal part of review content relates not to the content itself but to its formatting: the use of bullet-point lists, for example, is expected to increase review's helpfulness, as it delegates a localized space to each single argument, making it easier for the viewer to navigate, comprehend and memorize the contents. On the other hand, the use, and especially excessive use, of exclamation points “!” and successive opening (“(((((“, “(((“, “((((((((((((((((((((“) and/or closing (“)))))“, “)))“, “))))))))))))))))))))“) parenthesis This is a common way to express emotions on Russian social media and online forums. The parentheses are actually a multiple truncated smiles and frowns of old-era emojis (“:)” and “:(“) can annoy the viewer to the point that he/she will downvote a review. The lengths of these “symbol congas” can also indicate the seriousness of the author and, therefore, his or her review: if they are very long and common it is likely that a review is jokey in nature and thus having little potential for being helpful in making a real purchase decision. The effects of use of arbitrary “P.S.” segmentation is ambiguous, but it can be hypothesized that it might indicate underlying review's informativeness being so great, that some separate note had to be written out of the main text body in order not to confuse the viewer. If it can be used to indicate hyper-informativeness of a review, it should be positively correlated with its helpfulness as well.

Design, price and quality are features that are possessed by every existing product, so it should be included in our generalizing model, even if we don't fully understand if their mention in a review necessarily lead to it being more helpful. Another common feature is product's packaging and the fact that it has to somehow be delivered to the buyer. Delivery isn't linked to the product's inherit value, but is liked to the customer experience and the latter decides on the rating of a review to give and on praises/bashings to write. So, we might expect that any mentions of delivery or packaging in negative reviews are attributable to problems, unrelated to the product itself and therefore making it theoretically less helpful for other customers.

We must note, that a very significant portion of review helpfulness comes from the information, describing the actual qualities of the product it reviews (for example, loudness for earbuds and strength/healing properties for drugs). The effects it has on review helpfulness are incomparable between different types of products, as the selection of influential keywords differs as well. In order to account for all their individual sections of claims and the effect they have on review usefulness we would need to discover and deal with specific qualities and features of products in our samples. As we do not assess the effects of particular products' characteristics that can't be applied to other types of products, we will only focus on universal sections about product's metadata and the overall impression the user had with the product (sections described above). Those sections don't depend on the type of the product and so our models won't need to be tailored to include their unique characteristics.

1.3 Level of review oversaturation

Reading, analyzing and comparing reviews is work in itself and thus takes additional toll from the viewers. Both time and effort have to be spent to navigate through the reviews and this burden increases with the number of reviews a viewer decides to take in consideration. Therefore, it is reasonable to assume that there exists some sort of a cutoff number of reviews, beyond which marginal contribution of each additional review will be too insignificant (or even entirely nonexistent) to cover the efforts put in analyzing it. This assumption holds for top section of reviews sorted by helpfulness as well. No sane person will study all 100+ reviews and make paired comparisons for selection of that size. There needs to be a limit to the incoming information. While this limit is most likely dictated by personal preferences and individual user's endurance, we believe that we can identify an average number of reviews after which the helpfulness levels of all other reviews would gradually decrease. This relationship is explained with each subsequent review containing less and less additional information, as previous reviews are likely to support similar claims/topics. Although “excessive” reviews might still be useful if a viewer wants to validify claims made in reviews he had read before or to get a grip of what types of opinions prevail, they contribute little to product understanding. Thus, reviews that are posted after a certain saturation threshold are expected to have lower helpfulness ratio.

Ease of comprehension by the viewer

We have already touched on the effect technical complexity of the product has on its reviews in “review genuineness”, where we saw how more complex products require more thorough investigation of reviews by the customer. Product's complexity should be viewed in a bundle with other review qualities as it affects the strength of and somewhere even reverses the relationship between them and helpfulness. We expect review posting recency, presence of expertise claims by the author and number of words in a review to be affected in the following ways:

For technically complex products, helpfulness of more recent reviews should be higher than for similar reviews Ceteris paribus of unsophisticated products. If the product is complex then it would follow that it is more likely to be frequently updated, upgraded, redesigned or improved in any other way. The changes in technically complex products therefore can only be accounted for in newer reviews that were written after the changes in the product had been implemented. If both newer and older reviews are visible in the top section the older ones will be downvoted more heavily than newer ones as time goes on, because their claims will no longer be considered relevant/factually accurate.

For technically complex products, helpfulness of bigger, wordier reviews should be higher than for similar reviews of unsophisticated products. If the product is technically complex, it is likely that in order to properly describe his experience with it an expert author might want to utilize his specialist vocabulary in a form of technical jargon. If a reviewer uses some sort of technical jargon in his review, regular customers, who lack specialist knowledge to understand its meaning or implications will not benefit from the portion of the review pivoting on that lexicon. Message of the review has to be simple or clearly explained, otherwise its helpfulness will be lowered. As a side observation, we also expect that the length of the review can be used to indicate if jargon explanations are present within the text: if the review is longer and contains technical jargon it is likely that extra length is attributable to necessary explanations, examples and case studies. In such cases the use of technical jargon will not negatively affect the ease of comprehension of review information by regular viewers. Those reviews will not have lowered helpfulness.

For technically complex products, helpfulness of reviews containing expertise claims should be higher than for similar reviews of unsophisticated products. When dealing with a technically complex products, customers may favor trusting the expertise of authority figures, rather than trying to become deeply knowledgeable in the subject themselves, in order to save time and effort. For simpler products or for the products that a customer has already had experience with presence authority claims will have lesser impact on review helpfulness, primarily because people want to hear coherent, logical arguments that they can follow when they deal with something they can recognize.

Research questions (summary)

Table 2 Research questions.

Research Question	Why Tackled
1. To what extent are reviews neglected in Russian online market communities? (assume we speak exclusively about Russian online markets in all our RQs)	To understand the severity of neglection problem. To see what reviews are most undervoted. Is needed to justify the relevance of studying our chosen sample, validation of theoretical background and for future comparison with other markets.
2. What factors contribute to review helpfulness?	To set up a predictive model and create a list of criteria by which we will later rearrange the reviews. Is needed to analyze the impact of various review qualities on helpfulness and create a better sorting algorithm
3. Can our model improve customer reviewing experience?	To see the extent of improvements our changes will bring. Is needed to assess model's applicability.

Research Design

Hypotheses.

Table 3. List of hypotheses.

№	Hypotheses of RQ #1
1	Russian online reviews have review helpfulness rating distributions similar to those in Western markets: bell-curved with a very strong right skew (Assume that we speak about Russian online market community from now on)
2	0 votes reviews generally strongly outnumber 1+ vote reviews
3	0 votes reviews strongly outnumber 1+ vote reviews no matter the product type
4	Negative reviews are much more popular Popular reviews receive more votes than positive and neutral reviews
5	After a certain number of reviews has been posted, each successive review gets less votes on average Less than the reviews, posted before the oversaturation threshold had been passed
6	Top review section, sorted by helpfulness doesn't fairly represent proportions of positive, neutral and negative reviews of the overall review base
7	Recent reviews are less popular than the older ones
8	Reviews, in which the conclusive section is longer than argumentative section, are more popular than those, in which this relationship is reversed
9	Reviews of authors, whose purchases are verified are more popular than those, that come with only a claim of purchase
10	Reviews of anonymous authors are less popular than reviews of registered authors
11	Reviews of authors with non-default avatars are more popular than of those with the default avatars
12	The more images an author includes in his review - the more popular it becomes
13	The higher is author's expertise with the product, the higher are the scores for the respective reviews
14	Shorter reviews are more popular than longer ones
15	Reviews with bullet point format of “pros” and/or “cons” sections that lack description are less popular than extensive, argumentative bullet point reviews
	Hypotheses of RQ #2
16	Russian online market community generally sees extreme online reviews as more helpful than moderate ones
17	Negative reviews are more helpful than positive ones
18	Shorter reviews are more helpful than longer ones
19	Reviews, in which the conclusive section is longer than argumentative section, are more helpful than those, in which this relationship is reversed
20	Reviews that have every structural element (“pros”, “cons” and a “comment”) are more helpful than others. Reviews that don't have ANY segmentation are the least helpful.
21	Reviews that feature bullet-point arguments are more helpful than others.
22	Reviews with bullet point format of “pros” and/or “cons” sections that lack description are less helpful than extensive, argumentative bullet point reviews
23	Recent reviews are more helpful than the older ones
24	After a certain number of reviews has been posted, new reviews become less helpful than the reviews, posted before the oversaturation threshold had been passed
25	Reviews of authors, whose purchases are verified are more helpful than those, that come with only a claim of purchase
26	Anonymous reviews are less helpful than reviews of registered accounts
27	Reviews of authors with non-default avatars are more helpful than of those with the default avatars
28	The more images/photos an author includes in his review - the more helpful it becomes
29	The higher is author's expertise with the product, the more helpful his review becomes
30	Reviews with two-sided argumentation are more helpful than one-sided reviews
31	The use of technical jargon (i.e., abbreviation, complex concepts and uncommon terms) decreases review helpfulness
32	If an author claims to be an expert in a relative field, his review becomes more helpful
33	Review's helpfulness increases with the number of arguments it contains
34	The less words an author uses to deliver a single claim, the more helpful his review gets
35	Reviews that feature product comparisons are more helpful than those that don't
36	Reviews that feature price/quality analysis are more helpful than those that don't
37	Reviews that test the truthfulness of product description are more helpful than those that don't
38	Reviews that feature summarized reviewer's opinion are more helpful than those that don't
39	Reviews that feature a conclusive statement are more helpful than those that don't
40	Reviews that feature an indicator of ordinal product's quality measurement are more helpful than those that don't
41	Reviews that directly mention the price of the underlying product are more helpful than those that don't
42	Reviews that directly mention the quality of the underlying product are more helpful than those that don't
43	Reviews that directly mention the design of the underlying product are more helpful than those that don't
44	The presence of images or photos has even greater positive effect on review helpfulness, if the underlying product is expensive
45	The presence of a verified purchase's attestation has even greater positive effect on review helpfulness, if the underlying product is expensive
46	Featuring of price/quality analysis contributes even more to review helpfulness, if the underlying product is cheap
47	Featuring of product comparisons contributes even more to review helpfulness, if the underlying product is cheap
48	Praiseful (extremely positive) reviews become more helpful, if its author holds on with writing the review (has higher level of expertise)
49	Long reviews become more helpful, if its author holds on with writing the review (has higher level of expertise)
50	Recent reviews are more helpful than the older ones, if the underlying product is technically complex
51	The presence of an expertise claim has even greater positive effect on helpfulness, if the underlying products is technically complex
52	Longer reviews are more helpful than shorter ones, if the underlying product is technically complex
53	The number of arguments is a better extensiveness predictor of helpfulness than the number of words in a review
	Hypotheses of RQ #3
54	With an improved sorting algorithm, customers will see less reviews that they would consider unhelpful than with the current score-based algorithm (it is possible to detect and eliminate bad reviews based on their features alone).



2. Data Collection

2.1 Sources

For the purposes of answering our first two research questions, review data alone will suffice. We will extract it from “Yandex.Market.ru” using web scraping tool BeautifulSoup4 from the library of the same name that is run with Python.

We will use regular expression functions to identify whether the text of a review contains keyword indicators of selected qualitative content variables. To build these expressions we will need to understand what keywords could be used to convey certain ideas and how can they be represented as a part of a coherent sentence. We will unite these keyword regular expressions in dictionaries, that you can see attached in the Appendix. The composition of dictionaries will be supported with “Wordhelp.ru”, “Text.ru” and “Synonymonline.ru”. With the help of these resources, we account for many possible synonyms and writing variations that authors might implement, as well as get cognizant about the unconventional use of writing elements that would change keyword's meaning from what we might expect.

To answer our third research question, survey replies from 13 respondents will be used to assess the accuracy of our predictions.

Samples

When dealing with absolute visibility scores (relevant to the first research question), we will need to analyze only the reviews of equally popular products. Therefore, we will need to sample reviews data from products of different popularity levels (with different total number of reviews) and compare between them. Collectively, they will form a sample that we will call “Visibility sample”. Visibility sample will have 103,935 observations in total and will be used to identify the relationships between various review features and its visibility in Russian online markets. Random sampling was not necessary, because we literally collected the review data on more than half of Yandex.Market's products and on all of them that had more than 100 reviews in every product category.

To answer our second research question, we decided to only analyze reviews with at least 30 cumulative up- and downvotes. This way, we ensure that enough people have voted on a review to justify our assumptions of sample and individual observations' representativeness. Product's popularity will not matter here, as all our observations will be generalized. To understand the effect of review features on its helpfulness, we will sample 9,038 observations and call it “Training sample 2” or “training helpfulness sample”. These samples will also be used to build an aggregate predictive model with all influential factors. “Test sample” or “test helpfulness sample” is a sample of 2,268 observations that is needed to validate that the model's significance isn't due to chance and to assess how it performs on a new data.

Chart 1.

Most recent sampled reviews had been posted the day they were sampled, while the oldest ones had been posted 18 years before sampling, creating a data time range of 18 years. Although, in this research time periods are completely irrelevant as we treat all observations the same way, save for the posting order within a single product.

In this study we will be sampling only the reviews on products (so no services or providers). Overall, we sampled from 759 products in almost all categories (excluding “18+” products). The list of sampled products is too massive to include in this paper, even as an attachment. Their distributions based on their typization, however, you can see below

Chart 2.

In the two tile graphs above, you can see just how big the samples of various product categories are and how many reviews does each include.

Image 1. Example of a review observation and indication of its relative features.



Observations

In the image below we summarized, what review section denotes what features (variables), so that you will have an idea, where we take information from. The main body of a review (big highlighted segment, below the pictures) will be our source for every unincluded qualitative variable, that will require manual evaluation.

Note: “Recency” has been depreciated, as it has too big gaps between the possible values, we'll use posting order (read further) to approximate the recency of a review

Methods

Our analytic process follows the structure of our research questions:

Table 4. Data preparation before analysis initiation.

Step	Tools	Process description	Purpose
1	Sorting tools of Market.Yandex,ru BS4 and URL libraries (Python)	We will search for generic product categories that we will have chosen beforehand and select most popular products with the appropriate number of reviews Then we collect reviews on those products and divide them into defined samples using the BeautifulSoup library in Python	To obtain all needed data samples
2	Data tables	Data table with counts of product types Products sampled for RQ #1 Products sampled for RQ #2 Products sampled for RQ #3	Data preparation (to make sure, we have every possible product type combination in our samples - at least 1 per unique type and enough reviews represent each category)
3	Dictionaries	Presence of each review section Address the “Review Informativeness” of the “Literature review and theoretical background” section will be identified automatically by a computer. To allow the computer to find them, we need to create a list of as many keywords for those sections as possible, including word combinations, synonyms and all variations/modifications of these keywords. We will use online dictionaries to create our lists of keywords. This process is somewhat subjective and therefore isn't cut-in-stone. We plan to only include close synonyms	To quantify qualitative features of a review, so they can be included into a regressive model
4	Histograms Descriptive statistics	Assessment of variability in our data (assessment of qualitative variables' variability will be held off until RQ #2 is tackled)	Data preparation (to make sure, our categoric variables have enough observations for every possible category and determining the range of our scale variables, that we will use to limit our model's predictive range)

Table 5. RQ #1 (methods).

Step	Tools	Process description	Purpose
1	Histograms (visual assessment)	Visual assessment of histograms of reviews' aggregate scores (review visibility) for: Products of different popularity levels Different product types	To see, if Russian online market platforms suffer from review obscuration The reviews of some product types are more susceptible to review obscuration
2	T-tests	t-tests between reviews of different star-ratings. To remove the effect of product's popularity, we will compare only reviews from the same products.	To see, if negative review are more popular than neutral or positive reviews on average
3	Histograms Center and spread comparison	Center and spread comparison between the distributions of star-ratings of reviews in 2 groups: top reviews and the entire review base	To see, if the top review section, sorted by helpfulness fairly represents proportions of positive, neutral and negative reviews of the overall review base
4	Scatterplots (visual assessment)	Scatterplots of visibility (number of votes on a review) on the Y axis and a quantified review feature on the X axis. This is done for each separate feature (variable).	To prescreen, if a relationship exists between the variables, what type is it, and to identify reviews, most susceptible to obscuration To determine the type of relationship for the time-related variables (the effect of levels of oversaturation and posting recency), which can include hidden trenches that break linear trend after a certain point (and to determine that point, if one exists)
5	Regression (type depends on scatterplot patterns identified in the previous step)	All regressions in this step have the following structure: Visibility score = a + (factori)*вi + … + [product_type_control_variablek]*вj + … + [(factor)*(factor)]r*вn + e This model is linear, but we will plot each independent variable against the dependent one before we begin to regress, to see the nature of relationship and to make fitting adjustments if necessary, We will do so for every regression we run Then we analyze в1: its p-value and R2 of the model	To test, if a relationship exists between the variables, what type is it, how strong it is independently of other factors and what its direction is

Table 6. RQ #2 methods.

Step	Tools	Process description	Purpose
1	Scatterplots (visual assessment)	Scatterplots of helpfulness (ratio) on the Y axis and a quantified review feature on the X axis. This is done for each separate feature (variable).	To prescreen, if a relationship exists between the variables and what type (to enable proper variable coding - flat, log-transformed, quadratic, etc.) To determine the type of relationship for proper factoring in the model (linear, log, quadratic, etc.)
2	Regression model (type depends on scatterplot patterns identified in the previous step)	All regressions in this step have the following structure: Helpfulness = a + (factor)*в1 + e Then we analyze в1: its p-value and R2 of the model	To test, if a relationship exists between the variables, how strong it is independently of other factors and what its direction is To simplify our model by discarding insignificant factors and excluding them from it
3	Regression model (same as above, but now with control variables for product types) T-test comparisons	[This step is conducted along with the previous one] All regressions in this step have the following structure: Helpfulness = a + (factori)*вi + … + [product_type_control_variablek]*вj + … + [(factor)*(factor)]r*вn + e Then we analyze в1 and в1: their p-values and R2 of the model p-values of variables in regressions and t-tests are analyzed	To see, how various product types affect the strength of discovered relationships
4	Linear regression	Our final regression will include all significant factors from the previous step and will have the following structure: Helpfulness = a + (factori)*вi + … + [product_type_control_variablek]*вj + … + [(factor)*(factor)]r*вn + e Where [(factor)*(factor)]r are special variables that account for interaction effects. These variables are first added at this step. We will be interested in magnitudes of various вs, their p-values and model's R2-adj Insignificant factors will be excluded from the final model.	To create a predictive model for review helpfulness that we will use to redesign the current sorting algorithm
5	Variance inflation factor analysis	All significant variables are tested on multicollinearity with VIF scores assessment. If any variable will have VIF higher than 10 it will be excluded from our model. This exclusion will be done automatically with a certain specification to our regression model.	To deal with multicollinearity issues
6	Linear regression	Out-of-sample testing of our model on a test helpfulness sample. We will assess the differences between the RMSE of training and test samples.	To validate our findings

Step	Tools	Process description	Purpose
1	Our model	A hundred products from the Test helpfulness sample are selected and filtered to only include those reviews, that do not have any votes on them (0-vote reviews). Then we run our model on those reviews and assess their expected helpfulness levels. We choose the expected best and the expected worst reviews and group them into a pair Overall, we have 100 pairs of hypothesized worst and best 0-vote reviews from a hundred different products and we do not forget to code their names as “the best” and “the worst”	Data preparation
2	Survey T-test comparison	These 100 review pairs are shown to 13 different respondents (we will distribute those 100 pairs equally among them), who rank them by their perceived (real) helpfulness. The respondents are kept ignorant of which review is coded “the worst” and which one is “the best”, so they vote on each review fairly. The ranking is relative, so even if both reviews are good or both of them are bad, one still has to receive an upvote and the other - downvote. No ties. We then compare the average “hits”, which are the cases, when “the best” review was correctly proclaimed to be “the best” for each respondent with the mean of 0.5 - a value one would receive, if review-naming (“best”/”worst”) was completely due to chance. We will use a t-test for that.	To evaluate our model's applicability and improvement potential. This step finds out, if our model can be used to detect outright bad reviews even before customers waste their time reviewing them. A victory is consistently successful division between the best and the worst 0-vote reviews



Variables

Control variables:

Table 8. Product typization.

Product's characteristic	Measured as	Bins	Categories
Popularity	The total number of reviews a product has:	<=50 >50-100 >100-200 >200	[unpopular] [somewhat popular] [popular] [very popular]
Technical complexity	Is based on our expert assessment of technical complexity of a product, but mostly we consider all computers, media devices, their accessories and some home appliances to be complex, while calling every other category “simple” or “uncomplicated”	-	[not technically complex] [technically complex]
Price category	Will be determined by the histogram of product price distribution in our samples. The distribution and the binning we chose can be seen below on Charts 3 and 4	~ <5 tsnd RUB ~ 5 - 20 thsd RUB ~ > 20 thsd RUB	[Cheap] [Midrange] [Expensive]
Subjectivity level To what extent is it a search/experience good? We will also often call this control variable a product “Type”	Per-case assessment based on our expert judgement (this typization is subjective in nature)	[a qualitative variable]	[Search] [Experience]

Chart 3.



Chart 4.

Dependent variables:

Table 9. Dependent variables

Variable	ID	Description	Data Type	Relevant RQ
Visibility	Visibility	Total number of votes on a review (both positive and negative)	Scale	[RQ #1]
Helpfulness	Helpfulness	Number of upvotes on a review divided by its total number of votes Only reviews with 30+ total votes (visibility scores) will be analyzed	Ratio	[RQ #2] [RQ #3]

When it comes to independent variables, not all review features can be evaluated with 100% certainty. Those that can be, we will call “quantitative” or “objectively evaluated”, because their evaluation process is collectively exhaustive and no ambiguity is present. Other demand varying degrees of expert judgement calls and extensive search of specific keywords that could be used to encode qualitative text information (forming many keyword dictionaries). For some of those variables, we have a higher degree of certainty, connected to simpler or shorter keyword dictionaries necessary to quantify them. We refer to them as “semi-quantitative” or “semi-objectively evaluated”. The evaluation of the rest of the variables is up for debate and a lot of judgement calls and assumptions will have to be made to simplify the evaluation process. These variables are referred to as “subjective” or “subjectively evaluated”. We place keyword dictionaries for non-objectively evaluated variables in an appendix.

Objectively evaluated independent variables:

Table 10. Objectively evaluated independent variables

#	Variable	ID	Description	Data Type
1	Extensiveness	word_count	Total number of words in a review	Scale
2	Hyper-long review (Extensiveness grouping variable)	word_threshold_passed	Is needed to control for the especially large comments, that are expected to receive even less attention or have lower helpfulness on average word_count > 400 = 1 otherwise = 0	Binary
3	Rating	stars	Star-rating ranging from 1 (poorest product performance) to 5 (outstanding p.p.)	Ordinal
4	Extremity	extremity	Measured as an absolute distance between the neutral rating of 3 stars and the number of stars in a target review Possible values: 0, 1, 2	Ordinal
5	Divergence from the public opinion about the product (the aggregate product's rating)	status_quo_divergence	The number of stars that a review's star rating deviates from its underlying product's aggregate star rating (so, for example, a 2-star review for a [4.5]-star product will have status_quo_divergence = 2.5)	Scale
6	Reviewer Status	verified	Does the user have a “Проверенный пользователь” (“Verified user”) icon next to his account name? The icon is present = 1 Otherwise = 0	Binary
7	Posting Recency	posting_order	How many reviews had been posted before this one + 1	Scale
8	Helpfulness position	helpfulness_position	How many reviews are shown to the viewer, when he/she sorts them all by “helpfulness” before they witness this particular review? Is used exclusively for [RQ #1]	Scale
9	Image Count	images	How many images/photos a review contains	Scale
10	Reviewer has a non-default avatar photo	default_avatar	Is reviewer's icon different from the default one? Different icon = 1 Otherwise = 0	Binary
11	Reviewer identity is hidden	anonymous	Is review anonymous? Anonymous review = 1 Otherwise = 0	Binary
12	Level of Oversaturation	oversaturation_point_passed	Will depend on a certain value, determined during the analysis process Within a range (below the oversaturation value) = 0 Outside the range (above the oversaturation value) = 1	Binary
13	Level of reviewer's experience with the product (Level of Expertise)	user_expertise	Duration of time that the product had been in use by the reviewer at the moment of writing a review Is categorized by the user. Has following options: 1) Less than a month 2) Several months 3) More than a year	Ordinal
14	Fully segmented reviews	every_section	A review contains all 3 section - “pros”, “cons” and “comment” = 1 Otherwise = 0	Binary
15	Unsegmented reviews	not_segmented	A review is a solid wall of text, without any segmentation - neither of “pros”, “cons” or “comment” are explicitly present to guide = 1 Otherwise = 0	Binary
16	Section weight	longer_comm	The proportion of words in the “Comment” section of a review to the total number of words	Ratio

Semi-objectively evaluated independent variables:

Table 11. Semi-objectively evaluated independent variables.

#	Variable	ID	Description	Data Type
1	One-sided argumentation	twoSided_argumentation	Even when an author fills every review section, he/she might fill it with unargumentative statements (like “no” or “didn't find” in the “cons” section when the author is absolutely satisfied with the product). This variable catches these cases of de-facto lack of sections If either of review's pros or cons sections doesn't contain keywords from { DICTIONARY == twoSided_argumentation} or are left blank = 0 Otherwise = 1	Binary
2	Digit-style bullet points (BP)	any_digit_BP	If a review contains a bullet point-style list in any of its sections and math symbols are used as bullet points themselves = 1 Otherwise = 0	Binary
3	Math-style bullet points (BP)	any_math_BP	If a review contains a bullet point-style list in any of its sections and any consistent emojis are used as bullet points themselves = 1 Otherwise = 0	Binary
4	Emoji-style bullet points (BP)	any_emoji_BP	If a review contains a bullet point-style list in any of its sections and digits of various formatting styles are used as bullet points themselves = 1 Otherwise = 0	Binary
5	BP-lists	any_BP	Does a review feature any bullet point lists of any style in any of its sections? Yes = 1 No = 0	Binary
6	No-description bullet points	pros_average_words_per_BP	The average number of words per bullet point in “pros” section	Scale
7	No-description bullet points	cons_average_words_per_BP	The average number of words per bullet point in “cons” section	Scale
8	P.S.	P.S.	The presence of an arbitrary “P.S.” section at the end of a review This variable is measured with detecting a set of specific strings in the body of the review. The relating dictionary that includes all these strings can be found in the Appendix under the name {DICTIONARY == P.S.} A string is present inside the body text of a review = 1 Otherwise = 0	Binary
9	Use of exclamation points	exclamations	The number of cases “!' is used in a review. Strings with repeating “!” (“!!!”, “!!!!!!!”, etc.) are regarded as a single case	Scale
10	The average excessive use of exclamation points	excessive_exclamations	The number of cases when “!” is consecutively repeated in the body of a review
11	Excessive use of exclamation points	average_exclamation_conga_length	The average number of multiple consecutive “!” grouped together (e.g., 5 for “!!!!!”) inside the entire body of a review. 0 if no strings of multiple consecutive “!” are present in a body of a review	Scale
12	Use of smiles	excessive_smiles	The number of cases “!' is used in a review. Strings with repeating “)” or “(“ (“)))”, “(((((((”, etc.) are regarded as a single case	Scale
13	Excessive use of smiles	average_smile_conga_length	The average number of multiple consecutive “)” or “(“ grouped together (e.g., 5 for “)))))”) inside the entire body of a review. 0 if no strings of multiple consecutive “)” or “(“ are present in a body of a review	Scale

Subjectively evaluated independent variables:

Table 12. Subjectively evaluated independent variables.

#	Variable	ID	Description	Data Type
1	Number of Claims	number_of_claims	Total amount of claims/assessments made about the underlying product in a review Measured manually, by reading and assessing 618 top reviews with 200+ visibility scores. Each claim is assumed to be 1 unique statement/fact about the product. Relatively low num...

Подобные документы

• Product Placement в российском книжном бизнесе

  История и причины для размещения product placement. Виды размещения product placement: визуальный; вербальный; кинестетический. Отношение читательской аудитории к размещению торговой марки в книгах. Плюсы и минусы российского книжного product placement.
  
  курсовая работа [40,9 K], добавлен 24.11.2010
  

• Исследование особенностей Product Placement как инструмента PR-продвижения в киноиндустрии

  Историческое развитие и современное состояние Product Placement. Скрытая реклама в СМИ. Практическое применение Product Placement как инструмента маркетингового PR в РФ. Социологическое исследование Product Placement в российском кино, его преимущества.
  
  курсовая работа [332,4 K], добавлен 09.06.2014
  

• Formation of the marketing mix company

  Research tastes and preferences of consumers. Segmenting the market. Development of product concept and determine its characteristic. Calculating the optimal price at which the firm will maximize profits. Formation of optimal goods distribution.
  
  курсовая работа [4,4 M], добавлен 09.08.2014
  

• Product Placement в отечественных книжных изданиях (на примере художественной литературы)

  Product Placement в книжных изданиях: виды, преимущества и недостатки. Характеристика отечественного рынка книжной продукции: основные игроки. Популярные жанры художественной литературы и авторы для размещения Product Placement, их целевая аудитория.
  
  дипломная работа [119,9 K], добавлен 19.07.2011
  

• Marketing plan on BMW

  Purpose of the Marketing Plan. Organization Mission Statement. The main strategies employed by BMW. Sales volume of automobiles. New records set for revenues and earnings. Current models of BMW. Product life cycle. Engagement in Celebrity Endorsement.
  
  курсовая работа [879,4 K], добавлен 03.05.2015
  

• Достоинства и недостатки Product Placement в российской киноиндустрии

  История развития и характеристика основных достоинств и недостатков Product Placement в российской киноиндустрии как рекламного приёма, заключающегося в использовании реального коммерческого бренда в качестве реквизита. Применение рекламного логотипа.
  
  курсовая работа [98,6 K], добавлен 06.01.2011
  

• Запоминаемость прямой рекламы и Product Placement как к способа воздействия на потребительское поведение

  Скрытая реклама, ее понятие, характеристики и виды. Product placement как разновидность скрытой рекламы и техника его эффективного применения, ее отличия от других видов рекламы. Правовые основы размещения Product placement в современной телепродукции.
  
  курсовая работа [895,1 K], добавлен 19.10.2010
  

• Современные стратегии РР в кино

  История возникновения, понятие, сущность рекламы. Классификация по типу размещения, в зависимости от объекта, преимущества и недостатки РР. Правовое регулирование Product Placement. Разработка стратегии применения для продвижения продукции компании ASICS.
  
  дипломная работа [1,0 M], добавлен 23.08.2017
  

• Скрытая реклама в современной телепродукции

  Общая характеристика скрытой рекламы как разновидности рекламного продукта, ее типология. Анализ Product placement, его содержание, механизмы воздействия на потребителя, сравнение с другими видами рекламы, характер использования в современном телевидении.
  
  дипломная работа [2,9 M], добавлен 23.11.2009
  

• Рынок рекламных услуг

  Мировой рынок рекламы: особенности, состояние. Основные тенденции развития российской рекламы. Product Placement (скрытая реклама) в фильме "Дневной дозор". Планирование рекламной кампании, опирающейся на образ упаковки Водка "Абсолют": абсолютный успех.
  
  курсовая работа [82,4 K], добавлен 17.11.2014
  

• Product placement

  Законность размещения брендов в художественных произведениях. Понятие Product placement, определение рекламы и спонсорства. Сравнительный анализ указанных технологий. Бренды - герои фильмов. Правовые основы размещения брендов в произведениях искусства.
  
  курсовая работа [29,7 K], добавлен 05.02.2009
  

• Технология Product placement в компьютерных играх

  Исследование особенностей функционирования и развития Product Placement в компьютерных играх. Определение законности размещения торговой марки, товара или услуги в компьютерных играх с целью получения прибыли. Виды скрытой рекламы в компьютерных играх.
  
  курсовая работа [1,9 M], добавлен 17.06.2013
  

• Аналіз використання product placement у формуванні споживчої поведінки

  Product placement як різновид прихованої реклами і техніка його ефективного застосування. Правові основи розміщення брендів у творах мистецтва. Практичний аналіз художніх фільмів за наявності реклами. Розміщення певної торгової марки або самого товару.
  
  курсовая работа [2,3 M], добавлен 19.04.2015
  

• Теоретические основы формирования международной маркетинговой стратегии

  Классификация маркетинговых стратегий, основы их формирования. Компоненты 4Р (product, price, promotion, placement) в маркетинге. Политические, экономические, социально-культурные и правовые факторы, определяющие международную маркетинговую стратегию.
  
  контрольная работа [32,4 K], добавлен 08.01.2017
  

• Product Placement как способ продвижения гостиничных услуг

  Анализ особенностей технологии и основных каналов размещения Product Placement. Оценка эффективности использования данной рекламной технологии. Разработка проекта телешоу для продвижения гостиничных услуг. Исследование общественного мнения о телешоу.
  
  дипломная работа [261,0 K], добавлен 16.06.2013
  

• Procedure of preparation business-plan

  Business plans are an important test of clarity of thinking and clarity of the business. Reasons for writing a business plan. Market trends and the market niche for product. Business concept, market analysis. Company organization, financial plan.
  
  реферат [59,4 K], добавлен 15.09.2012
  

• Marketing research and development of Nestle company

  The concept of brand capital. Total branded product name for the whole company. Nestle as the largest producer of food in the world. Characteristics of technical and economic indicators. Nestle company’s brands. SWOT-analysis and Nestle in Ukraine.
  
  курсовая работа [36,2 K], добавлен 17.02.2012
  

• Travel agency "Play. Never stop"

  Plan of marketing. Absence of a large number of competitors. The emergence of new competitors. Active advertising policy. The absence of a known name in the market. Break even point. The price for advertising. Distribution channels and pricing policy.
  
  презентация [2,6 M], добавлен 15.02.2012
  

• The online advertising and its use in the World Wide Web

  A detailed analysis of lexical-semantic features of advertising in the World Wide Web. Description of verbal and nonverbal methods used in online advertising. Bringing a sample website hosted on its various banners and advertisements to consumers.
  
  дипломная работа [99,7 K], добавлен 10.04.2011
  

• Market Research. Market Research Methods

  The collection and analysis of information with a view of improving the business marketing activities. Qualitative & Quantitative Research. Interviews, Desk Research, Test Trial. Search Engines. Group interviews and focus groups, Secondary research.
  
  реферат [12,5 K], добавлен 17.02.2013
  

• главная
• рубрики
• по алфавиту
• вернуться в начало страницы
• вернуться к началу текста
• вернуться к подобным работам