Limit theorems are the theoretical foundation of statistical inference. Motivated by applications in statistics and machine learning we extend those beyond classical settings. Firstly, we consider empirical averages on distributionally "structured" random objects- i.e random objects whose distribution is invariant under the action of an amenable group. We show, under mild moment and mixing conditions, a series of universal second and third order limit theorems: central-limit theorems, concentration inequalities  and Berry-Esseen bounds. Beyond empirical averages, we consider general functions of exchangeable structures, for which we provide-under mild stability conditions general central-limit theorems. The utility of these will be illustrated by a series of example in machine learning theory, network theory and statistics.