At the end of the 19th century there was a major disconnect between science and philosophy. Since the publication of Kant's Critique of Pure Reason in 1781 they had been heading in different directions. In philosophy the golden age of German Idealism had culminated in Hegel's grand metaphysical system, while science had been through its own golden age of discovery of the mechanistic workings of the material world. These two ways of understanding reality are at odds with each other. The scientific view was that reality was made of material stuff, the behaviour of which could be completely reduced to mathematical laws. From the point of view of the idealistic philosophers, matter had a subordinate existence within the ultimate realm of mind, and the behaviour of reality was, at least possibly or partially, irreducible to mathematical laws. It looked to many people like both of these ways of understanding reality were almost finished. Hegel presented his system as the culmination of philosophy (a view widely interpreted as implying that the major work of metaphysics was complete), while physicists were hopeful that physics was nearing completion – just some loose ends needed tidying up here and there. There was no reason to suspect a dramatic paradigm shift was brewing. Physics and philosophy were not just separated, but had almost lost contact with each other. Scientists had no reason to think about reality in Kantian terms of phenomena and noumena, and this generally caused them no problems.
One exception to this general rule had occurred in the formative days of atomic theory earlier in the 19th century. On this occasion the boundary between physics and metaphysics was tested by a disagreement between scientists over the material existence of atoms. In 1808 British chemist John Dalton had discovered that if each chemical element is given a standard weight, then they always combine in fixed ratios: water is one part oxygen to two parts hydrogen. Dalton took these constant ratios to be evidence of the actual combination of real physical atoms, and proposed that at the smallest level, the material world is made of atoms. Most scientists agreed with him, but a small minority rejected this claim on the grounds that it went beyond the facts – after all, nobody had ever seen an atom, or any direct evidence of the existence of real atoms. When in 1826 Dalton received the Royal Society's medal of honour from chemist Humphry Davy, Davy warned that the word “atom” should be taken to mean no more than “chemical equivalent”. He recognised Dalton's achievement in purely practical terms: a discovery about how to do science, rather than what reality is made of. French chemist Jean Baptiste Dumas agreed that the word “atom” had no legitimate place in chemistry, on the grounds that it goes beyond experience. German chemist Friedrich August Kekulé claimed that the entire debate belonged to metaphysics, and that the question of whether or not atoms actually exist has nothing to do with chemistry. In fact, the first direct human experience of atoms occurred the following year, when Scottish botanist Robert Brown observed pollen grains in water lurching about as they were randomly bombarded by atoms, but at the time it was assumed that “Brownian motion” had some sort of biological explanation.
Another German chemist – Wilhelm Ostwald – proposed an alternative to the atomic hypothesis, based on the laws of thermodynamics. He claimed that atoms were mathematical fictions, and that the base level of reality was pure energy. Bitter disputes followed between the “atomists” and “energeticists”. Ostwald gave a speech in 1895 with the title On Overcoming Scientific Materialism: “We must renounce the hope of representing the physical world by referring natural phenomena to a mechanics of atoms....Our task is not to see the world through a dark and distorted mirror, but directly, so far as the nature of our minds permits. The task of science is to discern relations among realities, i.e. demonstrable and measurable quantities...It is not a search for forces we cannot measure, acting between atoms we cannot observe.”
In the late 19th century science, the viewpoint of the majority of scientists was that everything that existed in the world could be reduced to two sorts of entities: matter (or energy) and fields. Both were assumed by scientists to be real. It made no apparent difference whether they were considered to be part of phenomenal reality or noumenal reality. Kant's Transcendental Idealism was philosophy, not science, and physicists were not trying to provide foundations for a science of mind. But note Ostwald's chosen vocabulary: “measure”, “observe”, “the nature of our minds”. Since then, the precise meaning of these words and their relevance to the foundational assumptions of physics have become central concepts in a battle over the nature of reality that is far from over.
There were two physical fields: gravitational and electromagnetic (physics was soon to add two more – the strong nuclear field, which binds atomic nuclei, and the weak nuclear field, which breaks them apart during radioactive decay). In classical physics, fields were understood as continuous distributions of force or energy throughout space, and they have very different ranges. Electric and gravitational forces follow inverse-square laws. Magnetic effects, being part of electromagnetism, have more complex distance dependencies. The three classical forces never reach zero: everything in the universe is attracted to everything else by gravity, even if the effect is infinitesimally small at great distances. The two modern forces only act over the extremely short distances that apply to atomic nuclei, which is why they were unknown until the 20th century.
From the viewpoint of classical physics, only two sorts of laws are needed to explain everything – those that govern the motion of matter, and those that explain the behaviour of fields in terms of matter. These laws are all completely deterministic – if you know everything about the current situation, then, at least in theory, you can know everything about the outcome. At the time, deism was a popular belief – the idea that God created the universe like a piece of cosmic clockwork, set it in motion and then left it to look after itself in a completely deterministic manner. This determinism was defined by Newton's laws of motion and his field of gravity, but the field laws of electricity and magnetism were not discovered until the 1860s, by Scottish physicist James Clerk Maxwell. This discovery would lead to the unravelling of classical physics.
Maxwell's laws combined electricity and magnetism – they were two forces, but reducible to a single field. Quite unexpectedly, these laws cleared up what had until then been a mystery about the nature of light. Fields are associated with matter – if you shake the matter, then you shake the associated field, and this sends waves radiating away from the location of the shaking, just as waves radiate from a pebble thrown into a pond. Maxwell's laws enabled physicists to calculate the speed that electromagnetic waves travel, and this perfectly matched the speed of light, which had already been measured. This led directly to the conclusion that light must be high-frequency electromagnetic waves, and Maxwell also correctly concluded that there must be electromagnetic waves of other frequencies. Sure enough, Heinrich Hertz discovered radio waves in the late 1880s and in 1895 Wilhelm Conrad Röntgen discovered X-rays. Classical physics appeared to be complete.
Or at least, almost complete, for there was a fly in the ointment, known as “the Black Body Radiation Problem” or “the Ultraviolet Catastrophe”. Black objects have no intrinsic colour, but they take on a colour when they are heated up. The colour of iron changes as the temperature rises – red hot, white hot, etc... Physicists wanted to know how to calculate the colour, now known to be an electromagnetic wavelength, from the temperature. This, they presumed, must have something to do with the matter in the black body shaking more violently as it heats up, which everybody assumed must follow Newton's laws (though we now know light is emitted by moving electrons, which don't follow Newton's laws). The problem was that if you do the mathematics, the prediction is that black bodies should glow bright blue regardless of the temperature.
Then in 1900 German physicist Max Planck made a breakthrough. Instead of theoretically allowing matter to vibrate at any frequency, he restricted the energies of oscillators to discrete values E = nhf, where E is the particle's energy, n is any integer, f is the frequency and h is a constant that is now named after Planck. This rule restricts the particles to a finite set of energies defined by the value of hf. This innovation was not intended by Planck to be representative of reality – it was a trick to make the mathematics simpler, and Planck planned to eventually get rid of it by making the constant zero, so that this finite set was so huge that the matter could once again vibrate at any energy it wanted to, or as near as makes no difference. However, when he set the constant to zero, the bright blue glow returned. This problem had a solution he was not expecting – it turned out that if he set h to one specific value (6.62607015×10−34), then the predicted colour perfectly matched experimental values. This constant later became known as the “quantum of action”, since it has the dimension of energy multiplied by time, which is known as “action” in classical physics.
Nobody realised that this was the thin end of a wedge that would soon break classical physics apart. The theory produced exactly the right answer, but nobody could make any sense of it, because it directly contradicted the Newtonian view of reality. Classical physics and Planck's theory were both mathematical, but everything in classical physics is analogue, like a vinyl record. It was as if classical physicists had been searching for the last record to complete their collection of long players, and now Planck had found the missing recording, but rather than a record, he had found a compact disc. Nobody realised this was heralding a revolution in physics, because nobody understood the implications. Planck's new theory had solved the black body radiation problem, but could only describe reality digitally.
Next section: Einstein's four papers in 1905 | Two-Phase Cosmology