Production of low-quality or faulty products is costly for manufacturing companies since it wastes a lot of resources, human effort, and time. Avoiding such waste requires the correct set of process control parameters, which depends on the dynamic situation in the production processes. Research so far mainly focused on optimizing specific processes using traditional optimization algorithms, mainly evolutionary algorithms. To develop a framework that enables real-time optimization based on a predictive model for an arbitrary production process, this paper explores the application of reinforcement learning (RL) in the field of process parameter optimization. Inspired by the literature review on both, production process parameter optimization, and RL, a model based on maximum a posteriori policy optimization that can handle both numerical and categorical parameters is proposed. A validation study conducted on data sets from production fields compares the trained model to state–of–the–art traditional optimization algorithms and shows that RL can find optima of similar quality while requiring significantly less time.