Though piracy accounts for only a small fraction of the general losses of the maritime industry, it poses a serious threat to maritime security because of the connections between organized piracy, and wider criminal networks and corruption on land. Fighting piracy requires monitoring waterways, harbors and criminal networks on land to increase decision makers' ability to predict piracy attacks, and to manage operations to prevent or contain them. Piracy surveillance involves representing and processing a huge amount of heterogeneous information, often uncertain, unreliable and irrelevant, within a specific context to detect and recognize suspicious activities, and alert decision makers on vessel behaviors of interest with minimal false alarm. The paper discusses the role of information fusion, and context representation and utilization in building an piracy surveillance picture.