The time-varying Vector Autoregressions of for example Primiceri (2005) and Cogley and Sargent (2005) seem to be overparameterized. They assume that the vector of time-varying coefficients follows a random walk with full rank covariance matrix. Principal component analysis, however, suggests that only a few factors are important in driving the coefficients. Therefore, we reduce the rank of the covariance matrix of the random walk. We rewrite the reduced rank model using an underlying factor structure which we can estimate much faster than the full rank model. Because the reduced rank model is not subject to overparameterization as much as the full rank model, we can include more variables and/or more lags.