This thesis contains four studies of the effects of disorder and randomness on strongly correlated quantum phases of matter. Starting with an itinerant ferromagnet, I first use an order-by-disorder approach to show that adding quenched charged disorder to the model generates new quantum fluctuations in the vicinity of the quantum critical point which lead to the formation of a novel magnetic phase known as a helical glass. Switching to bosons, I then employ a momentum-shell renormalisation group analysis of disordered lattice gases of bosons where I show that disorder breaks ergodicity in a non-trivial way, leading to unexpected glassy freezing effects. This work was carried out in the context of ultracold atomic gases, however the same physics can be realised in dimerised quantum antiferromagnets. By mapping the antiferromagnetic model onto a hard-core lattice gas of bosons, I go on to show the importance of the non-ergodic effects to the thermodynamics of the model and find evidence for an unusual glassy phase known as a Mott glass not previously thought to exist in this model. Finally, I use a mean-field numerical approach to simulate current generation quantum gas microscopes and demonstrate the feasibility of a novel measurement scheme designed to measure the Edwards-Anderson order parameter, a quantity which describes the degree of ergodicity breaking and which has never before been experimentally measured in any strongly correlated quantum system. Together, these works show that the addition of disorder into strongly interacting quantum systems can lead to qualitatively new behaviour, triggering the formation of new phases and new physics, rather than simply leading to small quantitative changes to the physics of the clean system. They provide new insights into the underlying physics of the models and make direct connection with experimental systems which can be used to test the results presented here.