While launching a competition, hosting companies are seeking to get the best possible solutions thus increasing the effectiveness of the platform. In this paper, we will measure the effectiveness of these activities by the strength of users that the company can attract for its purposes.Therefore, to make theimplementation of the goal of the work possible we are going to make analysis using various models of machine learning. The task that we have in its nature can be represented as a classification task, because essentially the paper is going to classify the users by their choices, by the decisions they have made and the available options that they have. To solve this classification task, we will use the following methods: conjoint analysis, k nearest neighbors' algorithm as well as our newly developed, modified knn method.All these methods and the underlying techniques that they are using belong to supervised machine learning algorithms family. Generally, though it is necessary to mention that they are other types of algorithms too, here we distinguish two families of machine learning models: supervised and unsupervised machine learning models. The main difference between these families of models is the way the task of the researcher has been formulated. In case of supervised methods, the researcher should find out the function that based on several input-output observations maps a vector of the several classes to the output of examples of the function(Akinsola, 2017).Meanwhile, under the unsupervised methods,there is no assumption that the researcher has prior knowledge about the vector of outputs. As it was already mentioned our case also assumes the usage of the models from the family of supervised methods. Hereafter in the remaining part of this section of the paper, we will discuss thetechniques that we used in the work.

Conjoint analysis is a well-known method that gained substantial popularity during the decades and is widely used in marketing research. It was originally developed back in the 1960s with the contribution of,to our mind one of the greatest statisticians of the time, Tukey. He with Luce proposed a model the purpose of which is to determine which features of the products and services are meeting the preferences of the customers(Dobney, Carlos, & Revilla, 2017). Generally, the conjoint analysisassumes conduction of some sort of surveys to collect the necessary information about the characteristics that companies are willing to test. By testing one must understand the determination of the influence of those characteristics on the consumers' decision-making process(Hauser).As an example of the characteristics, that might be an interest for the company, can serve such elements as a new design color on a web site or even a new button on the remote control for a TV.Though, there are numerous forms of conjoint analysis in this study we will use choice-based conjoint analysis because we are analyzing the choices of the users based on the parameters of the competitions and the relationship of those parameters to the final decision-making. It is remarkable that 80% of conjoint analysis studies apply this method in their research(Steiner & MeiЯner, 2018). As we just mentioned will not be an exception in this case as our research is also going to make a choice-based conjoint analysis. Though our case will be somewhat different from what has been done so far as it will not be a “sterile” experiment that will provide some options to the respondents and ask them to choose a various combination of features according to their tests. Instead of that, we will delegate this task to the real, outer world. Thus, we think that this will allow us to deal with the problem of respondents perception's distortion when they are starting to mark the choices regardless of their interest in them but based on some subjective psychological factors(Hundert, 2009). For example, the case when a person is starting to mark the features randomly because of bore or laziness is a rigid example of such psychological behavior.

A disadvantage of the choice-based analysis model that we face with is that the researcher usually does not have unlimited access to all the variables that can influence the process of decision formulation, therefore the model ought to deal with just a limited number of variables. In this case, we fall under this constraint. However, as almost any other researcher we should put up with this fact as we cannot deal with the problem because of the scarcity of the resources and the time.

The choice-based conjoint analysis uses a multinomial logistic model with heterogeneous parameters under its hood(Voleti, Srinivasan, & Pulak, 2017). Logistic regression is a lot like the ordinary least-squares regression, however, unlike the letter one, it does not make any assumptions on the continuity of the data. That is logistic regression can be used to solve classification tasks, where the researcher is willing to distinguish between different categories. For example, one can take into consideration the case when it is needed to predict if the basketball player will throw the ball dependent on some distance from the basket. The underlying core idea of the logistic regression is the same as in case of the least-squares regression: minimization of the loss function. However, in the case of the logistic regression, due to its nature, the estimate is constraint between values of 0 and 1. That is it can take as an argument any number from the whole real line, but the corresponding values must not exceed 1. Because of the usage of the multinomial logistic regression,the choice based conjoint analysis allows the researcher to interpret the output of the model, which is exactly what we are chasing for: a well-interpretable model(Sperandei, 2014). Fairly, speaking there have been written a considerable amount of papers that are discussing various issues connected to the logistic regression, therefore we think that it will be better to limit the discussion of this model by so.

It is understandable that being used for such a long time the way the analysis has been held has been affected by some modifications from numerous authors.There have been numerous experiments with the identification of the estimates that can establish the significance of the product features for customers, for instance, a new way of the assessment of the relevance of characteristics has been introduced that is using sophisticated methods such as support vector machines (SVM)(Maldonado, Montoya, & Weber, 2014). Though, this kind of approach can significantly help the researcher to make more accurate predictions however the usage of SVM has a significant disadvantage for our purposes, which is that the results of SVM are not well-interpretable, therefore, we cannot make further discussions of the outputs of the model. Another reason why we are restricted in the usage of this technique is that it is well recommended on surveys, however, our case is a bit different. For the purposes of our study, no surveys have been conducted, but other methods of data collection have been used, more thoroughly about that in the upcoming section of the paper will be discussed.

As we promised we are not going to constrain our research by performing only a choice-based conjoint analysis. Another method that we will use is the k nearest neighbors' algorithm. The algorithm has tremendous popularity among researches, partly because of its flexibility and partly because of its simplicity of implementation. The knn algorithm was even included in a list of top 10 most popular data mining techniques (Xindong, 2007).It is astonishing the number of various fields where knn algorithm finds its implementation starting from the music recommendation systems and finishing with the representation of superpixels for images(Kang, et al., 2018).At the same time, the idea behind knn is quite intuitive. Given a bunch of data points and a query point in an n-dimensional space, the algorithm finds the closest points to this query point and compute the estimate of our interest based on some estimates received from those nearest points (Beyer, Ramakrishnan, Goldstein & Shaft, 1997).The flexibility of the algorithm essentially lies in the fact that it can be successfully used during the resolution of both regression and classification problems. In fact, I with my colleague in the previous year, specifically in 2018, took the responsibility to modify the algorithm to make it more accurate during the solutions of regression's tasks. Uncovering some further parts of the work that will be discussed in the next section we developed a new modified knn algorithm to deal with the solutions of the classification tasks too.

One of the crucial factors that makes it for us possible to turn this idea into real life is the fact that knn, unlike other widely used models, makes relatively fewer assumptions about the underlying observations. Specifically, it does not make any assumptions about the distribution of the observations, thus giving an opportunity to the researcher to use the model even when he does not have any prior knowledge about the nature of the observations(Islam, 2010). Allof the above mentioned gives us opportunities to use the model for our analysis. However, like every other model, this one also has its disadvantages. The main limitation that we are going to concern in this paper is the tendency of knn to pay too much attention to the irrelevant features(Bafandeh & Bolandraftar, 2013). Though using some techniques as putting appropriate weights on the observations can help to partially cope with the problem up to some extent we are we are going to make a trial to improve this method even more by making manipulations with the cases when the model will face with almost evenly distributed data classes.

To sum up the section we are willing to mention that our choice of these two models have been justified by the nature of our task, the popularity of the models and as a consequence of the existence of very mature programming tools that will make the interaction with the models significantly less painful. Particularly, we will use these tools when dealing with the knn method. The modified knn will be developed and carried out from scratch on our own.

2. Statement of the research question

As it was already mentioned in the introduction objective that we decided to tackleis the development of the tools aimed at the maximization of the effectiveness of the competitions in the Kaggle platform. To justify the choice of such a specific goal, it can be noted that nobody had done extensive research about this topic or in this field. Moreover, the popularity of the platform and the fact that it can serve to companies as a key to solutions for various data science and machine learning tasks makes the perspective of such kind of work as this paper even more attractive. Eventually, the companies who will be eager to use the results of the analysis that the paper suggests will be able to optimize their resources and identify the most vital characteristics of the competitions, to acquire the best specialists of the platform.

The research question of the paper is the following: “Are there any specific competition's characteristics that can have a significant influence on the choice of the users, keeping all other factors equal?” To answer this question and reach the objective of the paper we are going to solve the following five tasks:

1) Retrieve relevant data from web resources.

2) Create a model which will realize the model on datasets.

3) Compare the model performance with the other benchmarking model performances.

4) Identify the core features of the competitions and their relationships with the contestant's choice.

5) Determine whether the contestants will be successful at the competition or not.

Specifically, as it can be seen from the order of these tasks, we will get the answer to our research question at the end of the analysis after getting all the data, developing a model and comparing the outputs. Strictly speaking, we can omit the third point from our tasks and still manage to answer the question. However, in that case, the answers that we get may not be the most “optimal” as they will not be evaluated on the best model. Therefore, we will not get rid of the third task.

The main points that we will be necessary to complete the tasks vary from one to another. To solve the first task of data retrieval we wrote a web scrapper program in the programming language Python. The web scraping is just another name for web data extraction. It is a data scrapping used for extracting data from websites(Boeing & Waddell, 2016).It is a quite popular way of data collection which is used in those cases when the web page of interest does not provide some way of interaction mechanism such as application programming interface (API) to make it easier for developers to get the necessary information. Though, Kaggle web site has an API we found it very limited in its capabilities. Therefore, we decided to write our own program to deal with this task.

The second task is a lot more difficult since it demands not only the ability to write a program on some programming language but also a deep understanding of the techniques, that we are going to use to meet the objective of the paper. Therefore, in the theoretical part of the work, we attempted to give at least some general acknowledgmentwiththe methods that we are going to use and develop further in the research section. Since it is a very ambitious task to try to tune all the models that we are going to use in the purposes of the research we are limiting ourselves only with the one method: knn. Another problem in our way in order to bring into the life the second task is the fact that we must not only know the restrictions of the methods that were described in the theoretical part but also we should be able to cope with them and suggest new ways to stretch these limits.

Naturally, here comes the third task and the need to choose some matric upon which we will analyze the performances of the models. If we were solving regression problem, we will use such metrics as the residual sum of squares (RSS). The RSS has beenquite popular because it is quite easily computable and understandable. The name of the RSS comes from the fact that it isthe root squared sum of the differences of the so-called “real” and predicted values for the n-th variable. However, in our case, the usage of such metric is inappropriate because in case of classification tasks the “real” and predicted values will be a Boolean value, that is we will get a sum of a bunch of zeros if the prediction is right and a sum of a bunch of ones if it is not right. This kind of technique to assess the performance of the model is not very informative. There are other metrics that are used specifically for classification type tasks. This paper is going to use a metric called accuracy. However, we will not stop solely on the accuracy of the model and will give the whole output of it with the help of a confusion matrix. The confusion matrix represents a table of the predicted and the true variables. It makes convenient to compare the model output and see for example, how many true positive or else said truly predicted variables we have. It also gives an opportunity to asses not just the accuracy of the predictions but also computes the sensitivity and specificity of the model(Sunasra, 2017).

After the analysis has been conducted, at the fourth and fifth tasks we will identify those characteristics of the competitions and users that mostly contribute to the general popularity and success of them. For our purposes,we are going to assess the contribution of the competitions features on the popularity of the competitions and the user's success in competitions given their own set of characteristics. As it was mentioned earlier to do so we are going to use choice-based conjoint analysis which was described in more details in the theoretical part of the paper along with knn and our new modified knn.

Overall, the methods that we are going to use should be enough to fill the gap and complete the objective of the paper. It is possible to make this kind of claim because the models were specifically chosen for the implementation of the task of the paper. Though it is no surprise that more sophisticated methods of supervised machine learning exist, and they can be used here too, we are constraining ourselves with these models since they are easy to implement, quite popular and if we are speaking about choice-based conjoint analysis are very well-interpretable.

3. Research methods

In order to start our research and to solve the problem, at first, we must have a database to work with. The database that has been used in the study can be divided into two categories: primary and secondary data. The primary data comes from the Kaggle web page(Kaggle, 2019). There an interested person can find an extensive amount of information about the website. It is essentially, metadata of all Kaggle activities.

The process of retrieval of the secondary data was the hardest one. Though Kaggle provides API, which can be used to extract some information, it has somewhat limited capabilities for our purposes. Hence, unfortunately, the official Kaggle's API became useless for further work. Nevertheless, we found the solution from this situation. That is, we wrote a program on the Python programming language which is intended to extract all the meta information that we are interested in. Particularly, we were interested in the characteristics of the users of Kaggle. As a result of a very long process of data retrieval, we get the database that finally can be useful for our research. The database consists of such information as the level of competences in all the sections: competitions, discussions, and kernels. Moreover, it contains data about the user's achievements: rankings and medals, once again from all sections of the platform. The last but not least, it has information about what competitions the users have been involved in and how successful they have been in these competitions. The level of success in the competitions is determined by the place that they managed to reach in the leaderboard of the competition.

Due to the technique, that the whole database was collected, it was initially separated into small datasets. There were joined with each other to make the ongoing manipulations easier. Since, as it was mentioned in the theoretical part, one of the methods, that the paper will use is going to be the choice based conjoint analysis model, we altered the shape of the data, to make it more suitable for this type of analysis. The conjoint analysis assumesthat the datasetwill get the categoric output variables for every user, thus we inserted a column in the table which describes whether the user chooses to take part in the competition or no. The issue with this approach is that there are more than hundred competitions that are starting at different times, and as we are modeling the choice of the competition from the user perspective, we cannot just take all the competitions and assign to them a binary indication of participation. Instead of it, for every user, we should take only those competitions that have been launched before the user made his or her mind to join in the competitionand have not been canceled by the time, he or she made the final decision. For example, if a user is involved in “A” event, and it started in the March 5, then we should look only at the events that started in March 5 or earlier and which expire date is later than the March 5.

A kind of adjustment that we must make to meet the preconditions of conjoint analysis is that we must allocate data in a so-called “long” format. This kind of format assumes that the data will be compressed in a way which shows the choice the user made and next to it there will be an alternative that he or she had. So, commonly, the side observer can notice duplication of the information by rows in our dataset. This structure will help us to capture all the possible variants for the users and simulate their decision making while choosing the competition.

At the end of the subsection we would like to mention that appendix 1 illustrates the encodings of the variables used in our sample and explains their meaning.

3.2 The development of the methods

4.1 Exploratory data analysis