In this paper, building a rule-based fuzzy hierarchical model with information granules is accomplished. In this scenario, a number of separate sources of data and the resulting individual fuzzy models formed on their basis are encountered. The ultimate objective is to realize rule-based fuzzy modeling globally on the basis of invoking some mechanisms of knowledge sharing and reconciliation. The underlying format of knowledge built by individual systems is that of information granules and fuzzy sets in particular. Thus, a mechanism of generating information granules named the principle of justifiable granularity is explicated.