This paper studies linear quantile regression (QR) estimators in panel data settings with fixed effects. The estimation error in the fixed effects causes an incidental parameter problem in the parameters of interest, and we work out the first order asymptotic bias under an asymptotic where both N and T grow to infinity. This leading incidental parameter bias is of order 1/T, analogous to the situation in non-linear fixed effect panel models with smooth objective function. The key technical challenge in deriving our result is that the QR objective function is non-smooth, rendering the existing large T asymptotic bias results in the panel literature non-applicable. We provide analytic and Jackknife bias corrected estimators and study their performance in Monte Carlo simulations, and in an application to educational achievement of US high-school students.