Numerous counterterrorist activities on the Web have to distinguish between terror-related and non-terror related items. In this case Machine Learning algorithms can be employed to construct classifiers from examples. Machine Learning applications often face the problem that real-life concepts tend to change over time and some of the learned classifiers from old observations become out-of-date. This problem is known as concept drift. It seems to be doubly valid for terrorists acting on the Web, because they want avoid being tracked. This paper gives a brief overview of the approaches that aim to deal with drifting concepts. Further it describes, in more detail, two mechanisms for dealing with drifting concepts, which are able to adapt dynamically to changes by forgetting irrelevant data and models. The presented mechanisms are general in nature and can be an add-on to any concept learning algorithm. Results from experiments that give evidences for the effectiveness of the presented approaches are reported and discussed.