This paper presents an architecture for a system that assists workers in manual tasks. The system integrates different assistance modalities applicable for assembly and maintenance. It supports several operational environments and assists the worker by providing instructions for sequential processes. The system is not limited to specific assistance modalities and currently integrates augmented reality on multiple devices like smart glasses or tablets and enhanced sensor technologies to adapt assistance to the context. The instructions are formalized using entity-relationship diagrams and an information model focusing on AR-based assistance is developed. Finally, the system architecture used to integrate the different modalities in a modular manner is shown.