Optimizing Stock Price Prediction with LightGBM and Engineered Features

Abstract

Stock price prediction is a pivotal task in financial analysis, offering the potential to make informed investment decisions and improve risk management strategies. However, the complexity and volatility of financial markets pose significant challenges. Traditional statistical models, while useful, often fail to capture the intricate, non-linear dependencies inherent in stock price movements. Machine learning (ML) techniques have emerged as powerful tools for tackling these challenges, yet their effectiveness heavily depends on the quality of the input features and the chosen algorithm. Light Gradient Boosting Machine (LightGBM) has gained attention for its ability to handle large-scale data efficiently, model complex interactions, and provide fast training times. This paper explores the application of LightGBM to stock price prediction, emphasizing the critical role of feature engineering. By integrating temporal, technical, and sentiment-based features, the proposed approach demonstrates a significant improvement in predictive accuracy over traditional methods.