The advent of Industry 4.0 has revolutionised manufacturing systems, introducing unprecedented levels of customisation and variability. Traditional methods for controlling Work-In-Progress (WIP) often fall short in these dynamic environments, necessitating the development of adaptive and intelligent control strategies. This paper explores the application of Reinforcement Learning (RL) for adaptive WIP control in semi-heterarchical architectures for flow-shop production systems. We propose a novel framework that integrates RL, specifically Deep Q-Networks (DQN), with Discrete-Event Simulation (DES) to derive optimal control policies without relying on closed-form mathematical models. Preliminary simulation experiments demonstrate the effectiveness of the proposed approach in handling variations in job processing time variability and throughput reference targets, showcasing the merit and potential of RL for adaptive WIP control.