Deep Reinforcement Learning for Multi-Objective Optimization of Industrial Oven Scheduling
2024 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE credits
Student thesis
Abstract [en]
This research investigates the use of Deep Reinforcement Learning (DRL) to solve a multi-objective optimization problem (MOOP) in manufacturing. The research focuses on a simplified real-world scenario involving five ovens. The goal is to optimize both the utilization and capacity rates of the ovens by improving the sequence and batch of products processed. Different processing times and oven capacities complicate real-time decision-making for product dispatching, specifically the sequence and combination of products for each oven at each dispatch time. This increases the challenge of efficiency in DRL policy learning. Existing literature mainly uses evolutionary algorithms to solve MOOP, with few studies using DRL for optimizing Batching and Sequencing Problems (BSP), such as those for ovens.
In this study, we conducted experiments using standard Proximal Policy Optimization (PPO) and Deep Q-Network (DQN) models from a prior project and proposed a hybrid model that combines a standard PPO model with a parameter transfer strategy. This hybrid approach aims to enhance exploration and policy learning efficiency. The objective, defined as the average effective utilization rate (utilization rate multiplied by capacity rate), is decomposed into multiple subproblems using a weighted sum function. The neural network parameters are trained on one subproblem and transferred to neighboring subproblems, enabling the agent to explore from a good starting point for each subproblem. A simulation model serves as the environment for interacting with the DRL models. Based on 100 individual tests on unseen data for the hybrid model and comparison models (random rule, heuristic rule, and PPO and DQN models from a prior project), the hybrid model produces Pareto optimal solutions with a higher average effective utilization rate. Additionally, it reduces training time to 1/24th of that required by the previous PPO and DQN models.
Place, publisher, year, edition, pages
2024.
Series
SAMINT-MIA ; SAMINT-MIA-IEM-24002
Keywords [en]
Deep Reinforcement Learning, Multi-Objective Optimization, Decision-Making, Simulation and Optimization in Manufacturing
National Category
Other Engineering and Technologies
Identifiers
URN: urn:nbn:se:uu:diva-533002OAI: oai:DiVA.org:uu-533002DiVA, id: diva2:1882598
Educational program
Master's Programme in Industrial Analytics
Presentation
2024-05-29, Uppsala, 13:00 (English)
Supervisors
Examiners
2024-07-112024-07-052024-07-11Bibliographically approved