This report presents a novel methodology in evaluating performance of the popular on-line multilingual search engine AltaVista. We study some crucial aspects of natural language that usually disrupt translation process and the extend to which it influences retrieval results. Having prepared the test set, we analyze phenomena of an English and French language pair in relation to the strategy of browsing the Web for documents with the features specified in the query. Using Natural Language Processing techniques, we test a Machine Translation system performance in order to improve the translation quality, which in turn has an impact on the results of information retrieval in a language other than the query language.