Web-scraping Data on Small and Big Businesses on Amazon for Ranking Algorithms
Author: Pragya Jha
Faculty Supervisor: Luella Fu
Department: Mathematics
Amazon's marketplace boasts a significant presence of small businesses that contribute substantially to its sales. However, questions arise regarding the equitable representation of small businesses alongside their larger counterparts. This study delves into the ranking dynamics of small versus big businesses on Amazon's platform. We hypothesize that despite comparable product metrics, small businesses may not receive equal visibility in search results. Leveraging web-scraping techniques, we collected data on product attributes including ratings, reviews, and business size indicators. Employing statistical analyses such as p-value ranking and maximum likelihood estimation, we identified disparities in product rankings. Notably, small businesses often faced suboptimal placement despite similar performance metrics. Through this analysis, we unearthed shortcomings in Amazon's ranking algorithms, motivating the development of a novel algorithm aimed at fostering fair representation for all businesses. Our proposed algorithm incorporates diverse parameters and assigns varying weights to accommodate the unique needs of both small and big businesses. This research paves the way for future advancements in ranking algorithms, potentially integrating machine learning and artificial intelligence to enhance algorithmic intelligence and fairness in product rankings on e-commerce platforms.