We study a natural random walk over the upper triangular matrices, with entries in Z/mZ, generated by steps which add or subtract row $i + 1$ to row $i$. We show that the mixing time of the lazy random walk is $O(n^2m^2)$ which is optimal up to constants. This generalizes a result of Peres and Sly and answers a question of Stong and of Arias-Castro, Diaconis and Stanley. This is joint work with Allan Sly.