It is very important to ensure reliable performance of deep learning model for future dataset for healthcare. This is more pronounced in the case of patient generated health data such as patient reported symptoms, which are not collected in a controlled environment. Since there has been a big difference in influenza incidence since the COVID-19 pandemic, we evaluated whether the deep learning model can maintain sufficiently robust performance against these changes. We have collected 226,655 episodes from 110,893 users since June 2020 and tested the influenza screening model, our model showed 87.02% sensitivity and 0.8670 of AUROC. The results of COVID-19 pandemic are comparable to that of before COVID-19 pandemic.