One half of the 2021 Nobel Prize in Physics was awarded to Giorgio Parisi for his discovery of ‘hidden patterns in disordered complex materials’ which ‘make it possible to understand and describe many different and apparently entirely random materials’. But what does this really mean? Spin glasses are highly complex, intricate phases of matter which push our mathematical and computational techniques to their limit: fully understanding them remains a major open problem at the frontier of fundamental physics. In this talk, I will discuss the physics of spin glasses, describe how the insights of Parisi and collaborators opened the door to our modern understanding of these enigmatic magnetic phases, as well as how these techniques connect to modern fields as diverse as neural networks and machine learning. Finally I will outline some outstanding challenges in this highly active field, including my own research into potential links with many-body localization and applications towards quantum computing.