{"id":4283,"date":"2021-08-19T14:35:05","date_gmt":"2021-08-19T18:35:05","guid":{"rendered":"https:\/\/portfolios.cs.earlham.edu\/?p=4283"},"modified":"2021-08-19T14:38:37","modified_gmt":"2021-08-19T18:38:37","slug":"cs-388-the-first-three-pitches","status":"publish","type":"post","link":"https:\/\/portfolios.cs.earlham.edu\/index.php\/2021\/08\/19\/cs-388-the-first-three-pitches\/","title":{"rendered":"CS 388: The first three pitches"},"content":{"rendered":"\n<h1 class=\"has-medium-font-size wp-block-heading\">A new voting ensemble scheme<\/h1>\n\n\n\n<ul class=\"wp-block-list\"><li>Voting is a popular technique in Machine Learning to aggregate the predictions of multiple models and produce a more robust prediction. Some of the most widely used voting schemes are majority voting, rank voting, etc. I hope to propose a new voting system that particularly focuses on solving the issue of overfitting by using the non-conflict data points to inform the prediction of data points where conflict does arise.<\/li><li>Evaluation:&nbsp;<ul><li>Compare its overall performance with that of the popular voting schemes&nbsp;<\/li><li>Examine ties \u2192 see if it&#8217;s better than flipping a coin&nbsp;<\/li><li>Apply statistical hypothesis testing to these analyses<\/li><\/ul><\/li><li>Possible datasets to work with:<ul><li>Any dataset whose target is categorical (i.e. it&#8217;s a classification problem). Preferably, the features are numerical and continuous.&nbsp;&nbsp;<\/li><\/ul><ul><li><a href=\"https:\/\/www.semanticscholar.org\/paper\/A-Weighted-Majority-Voting-Ensemble-Approach-for-Do%C4%9Fan-Birant\/e2600cbc04da3284b61ef72223403f1dca3d2a98\/figure\/6\">The datasets used in the WMVE paper<\/a><\/li><\/ul><\/li><\/ul>\n\n\n\n<h1 class=\"has-medium-font-size wp-block-heading\">Comparing the performance of the different Hyperparameter Tuning methods<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">Hyperparameter Tuning is an important step in building a strong Machine Learning model. However, that the hyperparameter space grows exponentially and the interaction among the hyperparameters is often nonlinear limits the number of feasible methods to come up with a more optimal set of hyperparameters. I plan to examine some of the most common methods that are often used to tackle this problem and compare their performance:&nbsp;&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Grid Search<\/li><li>Random<\/li><li>Sobol (hybrid of the two aforementioned methods)<\/li><li>Bayesian Optimization<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">To many people&#8217;s surprise, the Random brute-force technique sometimes outperforms the Grid Search method. My project aims to verify this claim by applying the techniques above to a range of benchmark datasets and prediction algorithms.&nbsp;<\/p>\n\n\n\n<h1 class=\"has-medium-font-size wp-block-heading\">The referee classifier&nbsp;<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">This is another Voting ensemble scheme. We pick out the classifier that performs the best <strong>under conflict<\/strong> and give it the role of the referee to solve &#8220;dispute&#8221; among the classifiers. The same principle can be used for breaking ties but we can also try removing the classifier that performs the worst under conflict.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We can try out a diverse set of classification algorithms like Decision Tree, Support Vector Machine, KNN, Naive Bayes, Logistic Regression, etc. and run them on the <a href=\"https:\/\/archive.ics.uci.edu\/ml\/datasets.php\">benchmark datasets from UCI<\/a>. This proposed voting scheme can then be compared against the more common Simple Majority Voting Ensemble approach in terms of accuracy and other performance metrics.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A new voting ensemble scheme Voting is a popular technique in Machine Learning to aggregate the predictions of multiple models and produce a more robust prediction. Some of the most widely used voting schemes are majority voting, rank voting, etc. I hope to propose a new voting system that particularly focuses on solving the issue &#8230; <a title=\"CS 388: The first three pitches\" class=\"read-more\" href=\"https:\/\/portfolios.cs.earlham.edu\/index.php\/2021\/08\/19\/cs-388-the-first-three-pitches\/\" aria-label=\"Read more about CS 388: The first three pitches\">Read more<\/a><\/p>\n","protected":false},"author":66,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[19],"tags":[],"class_list":["post-4283","post","type-post","status-publish","format-standard","hentry","category-student"],"_links":{"self":[{"href":"https:\/\/portfolios.cs.earlham.edu\/index.php\/wp-json\/wp\/v2\/posts\/4283","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/portfolios.cs.earlham.edu\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/portfolios.cs.earlham.edu\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/portfolios.cs.earlham.edu\/index.php\/wp-json\/wp\/v2\/users\/66"}],"replies":[{"embeddable":true,"href":"https:\/\/portfolios.cs.earlham.edu\/index.php\/wp-json\/wp\/v2\/comments?post=4283"}],"version-history":[{"count":2,"href":"https:\/\/portfolios.cs.earlham.edu\/index.php\/wp-json\/wp\/v2\/posts\/4283\/revisions"}],"predecessor-version":[{"id":4285,"href":"https:\/\/portfolios.cs.earlham.edu\/index.php\/wp-json\/wp\/v2\/posts\/4283\/revisions\/4285"}],"wp:attachment":[{"href":"https:\/\/portfolios.cs.earlham.edu\/index.php\/wp-json\/wp\/v2\/media?parent=4283"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/portfolios.cs.earlham.edu\/index.php\/wp-json\/wp\/v2\/categories?post=4283"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/portfolios.cs.earlham.edu\/index.php\/wp-json\/wp\/v2\/tags?post=4283"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}