The terror and anti-terror activities on the Web pose a real challenge for web harvesting and data analysis tasks. The Internet virtual domain is likely to be used intensively by terrorist organizations. Using advanced innovative web harvesting AI-based technologies give a surprising advantage to those who master the Internet open source. In addition, developing and implementing a large-scale web harvesting intelligent terror & security IT system is an extensive and time consuming organizational endeavor. In order to ensure a successful system implementation resulting in the payoff of a technological intelligent terror and security IT system, the organization has to initiate and manage a large scale operation that consists of methodological, organizational, data integration and technological components. In this paper we present a comprehensive system to tackle this challenge. Starting with a theoretical web harvesting framework and leading to a collection of web technologies that were developed and which resulted in a complete product that gave the users vast field experience throughout the globe. The paper has two main objectives: present a comprehensive technological and methodological framework for developing and implementing a Web Harvesting IT security system; draw and discuss lessons learned and the conclusions from field experience gathered over the last decade. For achieving these goals we introduce and describe in detail an IT solution equipped with all necessary algorithms and tools, and then describe how this system and methodology was implemented in the intelligence community over the last decade. Lessons learned and real field experience conclude the paper with a discussion of the system's advantages in the fight against terror and terrorist attacks in cyber space.