Development of high precision systems for recognition of human actions directly from video records is still open problem. Recently, in smart environments the recognition of dynamic actions of human in motion receives a particular interest. We propose two approaches for human action recognition. In the first approach, the envelope of 30x30 pixels is applied to enclose invariant to dimensions human silhouette separated from background. Once the area with located figure is defined, the image sequence is used as input of convolutional neural network that extracts global figure features without previous image processing. The second proposed approach is based on natural knowledge of the human figure such as proportions of body and position of feet. Together with processing global features, we extract six local features combining in this way the holistic and cluster-based approaches for representation of human figure. The input sub-sequence of previously aligned binary silhouettes from video frames is processed to concatenate local and global features into a single feature vector feeding hierarchical system of three linear support vector machines for human action classification. In order to evaluate the proposed approaches, two frameworks for recognizing human actions such as walk, jump, run, side and skip have been designed and tested on Weizmann standard and proper developed datasets achieving correct classification rate of 97–100%.