Cancer screening and timely follow-up of abnormal results can reduce mortality. One barrier to follow-up is the failure to identify abnormal results. While EHRs have coded results for certain tests, cancer screening results are often stored in free-text reports, which limit capabilities for automated decision support. As part of the multilevel Follow-up of Cancer Screening (mFOCUS) trial, we developed and implemented a natural language processing (NLP) tool to assist with real-time detection of abnormal cancer screening test results (including mammograms, low-dose chest CT scans, and Pap smears) and identification of gynecological follow-up for higher risk abnormalities (i.e. colposcopy) from free-text reports. We demonstrate the integration and implementation of NLP, within the mFOCUS system, to improve the follow-up of abnormal cancer screening results in a large integrated healthcare system. The NLP pipelines have detected scenarios when guideline-recommended care was not delivered, in part because the provider mis-identified the text-based result reports.