The estimation of large Vector Autoregressions with stochastic volatility using standard methods is computationally very demanding. In this paper we propose to model conditional volatilities as driven by a single common unobserved factor. This is justified by the observation that the pattern of estimated volatilities in empirical analyses is often very similar across variables. Using a combination of a standard natural conjugate prior for the VAR coefficients, and an independent prior on a common stochastic volatility factor, we derive the posterior densities for the parameters of the resulting BVAR with common stochastic volatility (BVAR-CSV). Under the chosen prior the conditional posterior of the VAR coefficients features a Kroneker structure that allows for fast estimation, even in a large system. Using US and UK data, we show that, compared to a model with constant volatilities, our proposed common volatility model significantly improves model fit and forecast accuracy. The gains are comparable to or as great as the gains achieved with a conventional stochastic volatility specification that allows independent volatility processes for each variable. But our common volatility specification greatly speeds computations.