How good a particle physicist is GPT-3?

Recently I have been playing around with OpenAI's GPT-3 and I am very impressed by it. It reminds of the famous Arthur C. Clarke quote, “Any sufficiently advanced technology is indistinguishable from magic.” If you don't know what GPT-3 is, this is a good introduction with a lot of links.

Recently, a Google engineer has made the claim that one of Google's proprietary internal AI systems called LaMDA is sentient. A more useful perspective on such advanced statistical AI systems is to say that they are an extremely useful, wonderful illusion.

You can also talk to GPT-3, just create an account on OpenAI and go to the sandbox!

Here I will show a "conversation" with GPT-3 to gauge how good a particle physicist — or an illusion of a particle physicist — it is. Mostly these are questions one might ask a student in an exam situation. The questions are independent of one another, so they work with a context-free system such as GPT-3. By context-free I mean that GPT-3 does not maintain conversation state, each question and answer are indepedent from the AI's perspective.

GPT-3 is trained on large amounts of text downloaded from the Internet. So I expect it to do well on questions where there are high-quality answers on the Internet, and it "just" has to return it, in the correct english structure. For questions that require more reasoning and combining facts (from the question and training data) I expect to get less impressive answers.

Also see these previous Bytepawn posts on GPT-3:

• How good a Data Scientist is GPT-3? - Part I
• How good a Data Scientist is GPT-3? - Part II

Q&A with GPT-3

Q: The Standard Model is a gauge theory with a U(1) x SU(2) x SU(3) symmetry. Is this SU(2) symmetry the same as the SU(2) symmetry of a fermion's spin?

A: No. The SU(2) symmetry of a fermion's spin is an internal symmetry, while the SU(2) symmetry of the Standard Model is a gauge symmetry.

This is a reasonably good answer.

Q: In particle physics, why is the weak force SU(2) and not U(2)? Why is the strong force SU(3) and not U(3)?

A: The U(1) subgroup of SU(2) is not a symmetry of the weak force, and the U(1) subgroup of SU(3) is not a symmetry of the strong force.

Here, GTP-3 is not answering the question, since it's talking about U(1). A better answer would be:

The matrix representation of U(n) has dimensionality $n^2$ (ie. n2 linearly independent matrices), whereas SU(n) has dimensionality $n^2-1$. Each linearly independent matrix corresponds to a gauge boson. So a gauge theory that is U(n) has $n^2$ gauge bosons, a gauge theory that is SU(n) has $n^2-1$. Note that U(1) is the same as SU(1). It is an experimental fact that:

• electromagnetism has 1 gauge boson (the photon), so it's U(1)
• the weak force has 3 gauge bosons (W+, W- and Z), so it's SU(2)
• the strong force has 8 gauge bosons (8 gluons), so it's SU(3)

Q: In particle physics, why are we always using special unitary groups, like U(1), SU(2) and SU(3)?

A: There are many reasons for this. One reason is that special unitary groups are Lie groups, which means they are continuous groups with a well-defined algebra. This algebra can be used to construct representations of the group, which are useful for doing calculations in physics.

Another reason is that the special unitary groups are subgroups of the unitary group U(N), which is the group of all unitary matrices with determinant 1. The unitary group is important in physics because it is the group of all symmetry transformations that preserve the inner product of a vector space. The special unitary groups are important because they are the only subgroups of the unitary group that preserve the inner product up to a phase.

The special unitary groups are also important because they arise naturally in many physical situations. For example, the group SU(2) arises in the study of spin 1/2 particles, and the group SU(3) arises in the study of particles with color charge.

This is a reasonably good answer. On unitary operators, a better explanation would be:

Quantum mechanics (and QFT) is a probabilistic theory. When peforming measurements, there no certain outcomes, only probabilities. Furthermore, the "weights" associated with each outcome must be a complex number (complex amplitude), not a real number, because this is how we get our theory to reproduce the results of interference experiments (such as the two-slit experiment). So, the state of a quantum system is represented by a wave function, which consists of complex amplitudes, one for each possible measurement outcome. The sum of absolute squares of these amplitudes must sum to 1 — thus, we can imagine it like a unit vector in complex space. When we apply an operator to this state to (eg. the time evolution operator in the Schrodinger equation, or a gate in quantum computing, or a vertex in QFT), we want to preserve the property that the absolute squares sum to 1 — in other words, we want the operators to rotate around the unit complex sphere. Such operators are called unitary operators

Q: In particle physics, why does the strong force have 8 gauge bosons? There are 3 color charges, so why isn't there 3 gluons, one for each color charge?

A: The strong force has 8 gauge bosons because of the 3 color charges. The 3 gluons are the 3 color charges, so the strong force has 8 gauge bosons.

The answer is self-contradictory, in the first sentence it talks about 8 gauge bosons (the gauge bosons are the gluons for the strong force), but then in the second sentense it talks about 3 gluons. A good answer would be:

It is in general not true that there is one gluon for each charge. For the electromagnetic theory it's true, but it's an accident. Also for the weak force, where there are 3 gauge bosons, but there are not 3 weak charges. Also not that not all gauge bosons carry charge, examples: (i) QED's photon does not carry electric charge (or any charge) (ii) the weak force's Z boson does not carry any charge (iii) the W boson, although it's not part for QED (electron-proton-photons processes), carries an electric charge.

Q: What is the Hilbert space in Quantum mechanics?

A: The Hilbert space in quantum mechanics is the space of all possible states that a quantum system can be in.

The answer is not terrible, but could be more useful, like:

The complex vector space (with an inner product) where states describing a quantum mechanical system live (see above) is called a Hilbert space.

Q: In particle physics, what is the difference between helicity and chirality?

A: Helicity is a measure of the projection of the spin of a particle onto its momentum. Chirality is a measure of the handedness of a particle.

Good answer!

Q: In Quantum Field Theory, what is a current? What kind of currents are there?

A: In quantum field theory, a current is a conserved quantity associated with a symmetry of the Lagrangian. There are three types of currents in quantum field theory: vector currents, axial-vector currents, and tensor currents.

Good answer!

Q: In particle physics, what is the difference between the Cabbibo angle and the Kobayashi-Maskawa angle?

A: The Cabbibo angle is a parameter in the Standard Model of particle physics that governs the strength of the weak interaction between quarks. The Kobayashi-Maskawa angle is a parameter in the Standard Model that governs the strength of the weak interaction between quarks and leptons.

This is a bit of a trick question. There is such a thing as a Cabbibo angle, and Kobayashi-Maskawa are closely related, but there is no such thing as a Kobayashi-Maskawa angle; instead there is a Cabbibo-Kobayashi-Maskawa or CKM matrix. The CKM matrix describes the probability of a transition from one flavour quark to another flavour of quark, but not between quarks and leptons.

Q: What is the difference between the weak mixing angle and the Weinberg angle?

A: The weak mixing angle is a parameter in the Standard Model of particle physics that determines the strength of the weak force. The Weinberg angle is a parameter that determines the strength of the weak force in the electroweak theory.

This is a trick question — they are the same.

Q: In particle physics, for what kind of theories are Feynman-diagrams not useful?

A: In general, Feynman diagrams are not useful for theories that do not have a perturbative expansion. For example, in a theory with strong interactions, such as quantum chromodynamics (QCD), the Feynman diagrams become very complicated and are not very helpful in understanding the behavior of the theory.

Great answer!

Conclusion

Overall, 8 of the 10 of the GPT-3 answers are okay, which is very impressive. On some of the trickier questions, the answer is good english but incorrect or meaningless. This is the overall danger of GPT-3: if the reader is not an expert, s/he can't tell if they're getting a good answer (in this case, about half the time) or reasonable sounding non-sense.