We present a nonsmooth optimization method for solving the elastic contact problem. The Signorini contact problem is a variational problem that minimizes the elastic deformation energy subject to the contact inequality, i.e., the normal displacement at a given point of the boundary bounded above by an obstacle function. The Coulomb friction problem is a minimization of the deformable energy with a L¹ friction term at the boundary. We develop an effective numerical optimization method using the semi-smooth Newton method for the both variational problems. The method is of the form of Primal-Dual active set methods for Lagrange multiplier methods. We approximate these variational formulations with a multi-moment scheme based on Adini's elements which involves the use of the function values as well as the gradient values at nodes. The Primal-Dual active set method are then applied to these approximations. Finally we combine the solutions to the Signorini and Coulomb friction problems to solve the full contact problem.