Sign language machine translation (SLMT) aims to convert text to signs automatically for the benefit of deaf people. This paper proposes a new evaluation approach for SLMT. It aims to achieve good correlation between automatically-generated scores and human judgements of translation accuracy. An Arabic Sign Language (ArSL) corpus has been collected and used for evaluation experiments, which show the superiority of the proposed measure over others considered.