This paper develops a new efficient approach for multivariate time series data modeling and forecasting in the presence of an unknown number of change-points. The predictive density has a closed form by assuming conjugate priors for the parameters which characterize each regime. A Markov chain Monte Carlo method takes advantage of the conjugacyto integrate out the parameters which characterize each regime, treat the regime duration as a state variableand simulate the regime allocation of the data from its posterior distribution efficiently. Two priors, one is non-hierarchical for fast computation, the other is shrinkage hierarchical to exploit the information across regimes, are proposed. The model is applied to 7 U.S. macroeconomic time series and finds strong evidence for the existence of structural instability. A general pattern of the data is similar to the great moderation. However, we discover heterogeneous dynamics with infrequent volatility jumps for individual variables.The marginal likelihood comparison shows that our approach provides superior out-of-sample forecasting performance.