Associated to an abelian variety A/K is a Galois representation which describes the action of the absolute Galois group of K on the torsion points of A.   In this talk, we shall describe how large the image of this representation can be (in terms of a number field K and the dimension of A).   We achieve this by considering abelian varieties in families and then using a special variant of Hilbert's irreducibility theorem.     Some results of Serre on the mod ell Galois image will also be reviewed.    (This is joint work with David Zureick-Brown)