What are massless particles in layman's terms

Massless particles with spin do not have a "$ S_z = 0 $" state, since they actually have no spin like massive particles. They have Helicity which is the value of the projection of the spin operator onto the momentum operator. The reason for this is the representation theory of the group of spacetime symmetry, the Poincaré group.

To understand this, we must first remember that "spin" is the number that denotes irreducible representations of $ \ mathrm {SU} (2) $, the double coverage of the rotation group $ \ mathrm {SO} (3) . $. In the relativistic quantum field theory needed to describe photons, however, this rotation group is not the space-time symmetry group that we need to represent. Instead we have to look for representations of the identity-bound component of the Poincaré group $ \ mathrm {SO} (1,3) \ rtimes \ mathbb {R} ^ 4 $, ie the correct orthochronous Lorentz transformation together with translations.

Well, for the finite dimensional representations of the Lorentz group we are lucky that there is a "random" equivalence of algebra representations of $ \ mathfrak {see above} (1,3) $ and $ \ mathfrak {see below} (2) \ times \ mathfrak {su} (2) $, whereby we can identify the finite-dimensional representations in which classical relativistic fields are transformed by pairs of half-numbers $ (s_1, s_2) $, where $ s_i \ in \ frac {1} { 2} \ mathbb {Z} $ denotes a single $ \ mathfrak {see below} (2) $ representation. The actual rotational algebra is located diagonally in this $ \ mathfrak {see} (2) \ times \ mathfrak {see} (2) $, so that the physical spin of such a representation is $ s_1 + s_2 $. This determines the classic spin of a field assigned.

As is so often the case, quantum theory complicates things: Wigner's theorem implies that we are now standardized Have to look for representations of the Poincaré group in our Hilbert state space. With the exception of the trivial representation corresponding to the vacuum, none of the finite-dimensional representations is uniform (essentially because the Poincaré group is not compact and has no compact normal subgroups). So we have to turn to infinite dimensional representations, and here we do not have the equivalence between $ \ mathfrak {see above} (1,3) $ and $ \ mathfrak {see below} (2) \ times \ mathfrak {see below} (2) $ . The techniques used to realize this equivalence are based explicitly on the finite dimensionality of the representation. Especially there is no isomorphism like $ \ mathrm {SO} (1,3) \ cong \ mathrm {SU} (2) \ times \ mathrm {SU} (2) $, regardless of how often you will read similar statements in physics books. For more information on this topic, see e.g. this answer from Qmechanic.

It turns out that classifying the uniform representations is not that easy a task. The full classification is called the Wigner classification, and it turns out that for the construction of irreducible uniform representations it is relevant to consider the small group that corresponds to the momentum of a particle - the subgroup of the Lorentz group that leaves the momentum of the Particle invariable. For a massive particle this is $ \ mathrm {SO} (3) $, and it turns out that we can also characterize the uniform representation with our known spin $ s $.

For a massless particle the momentum $ (p, -p, 0,0) $ is not invariant under $ \ mathrm {SO} (3) $, but under a group called $ \ mathrm {ISO} (2) $ or $ \ mathrm {SE} (2) $, which is essentially $ \ mathrm {SO} (2) $ with translations. As an Abelian, $ \ mathrm {SO} (2) $ only has one-dimensional irreducible representations, which are characterized by a single number $ h $, which physically turns out to be the eigenvalue of helicity. There are more general cases for $ \ mathrm {ISO} (2) $, which are referred to as Continuous Spin Representations (CSR), but were not physically relevant up to now.

Well, that single number $ h $ reverses its sign under parity. For particles assigned to classical fields with a non-zero spin, we must use both the $ h $ and the $ -h $ representations. And that's it - massless helicity particles $ h $ have the $ h \ oplus -h $ representation in their state space, no spin representation of $ \ mathrm {SO} (3) $. The evaluation of the actual spin operator shows that our classical spin idea agrees with the number $ h $.

Without something To say about the photon or the electromagnetic field, we therefore know that massless particles have a spin other than zero two degrees of freedom to have. This is completely general and at the heart of the argument that all massless vector bosons gauge bosons are:

We know that a generic vector field has three d.o.f. - the independent field components that transform into one another under Lorentz transformation, hence three independent sets of creation and annihilation operators that transform into one another, hence we expect three different types of particle states.

But the two d.o.f. of a massless spin-1 particle do not fit - i.e. one of the d.o.f. a massless vector field must be "forged". The way d.o.f.s of fields are "false" is that the field is a measurement field and it is 1 d.o.f. with the freedom to choose a measuring device. The history of quantization of gauge theory - even in the Abelian case of electromagnetism - is subtle, and you rightly do not blindly accept the argument that the two classical polarizations of the gauge field - the longitudinal one eliminated by gauge symmetry - are different types of particle states in quantum theory: The decoupling of the states that one would naively associate with the longitudinal modes is ensured by the Ward identities and is not at all obvious a priori.

In this way, the properties of being a gauge boson and not having $ S_z = 0 $ and being massless are interrelated: Being one of these things immediately forces the other two as well. In this answer I considered "being massless" as the basic property, since it shows "no $ S_z = 0 $" without assuming anything more specific about the field - especially without being limited to measuring fields or electromagnetism a priori.