Physical manipulatives have been applied in traditional education for a long time. This paper proposes that by embedding computing power in manipulatives computers can monitor students' physical manipulations to support learning. This paper also describes the design of a digital desk prototype, called ArithmeticDesk to illustrate the vision of computer embedded manipulatives and takes learning fractions as an example. The study is an attempt to accommodate physical and virtual manipulations, as well as to eliminate the gap between traditional and computer-based learning activities. More experiments and studies will be conducted in the future.