In this paper, we address the problem of sequential pick-and-place with multiple objects when a specific goal configuration is given. The sequence of pick-and-place can generally be optimized using search-based methods. However, when the number of objects increases, there remains a challenge due to the exponential growth in computational complexity. Especially, the most challenging aspect is that a significant number of sequences are infeasible, leading to extended search times. In this regard, we propose an approach that efficiently addresses this issue by considering both pick and place aspects simultaneously while searching grasp poses with learning-based techniques to find feasible solutions expediently. In our experiments, the proposed method showed an enhancement of up to 90% in the average quality of trajectory for rearrangement benchmarks and about a 50% improvement in computational time in certain scenarios.