Modern CPUs are very complex devices, with billions of tiny transistors embedded on the chip. While this is great for getting things done, it makes it very difficult to understand how the CPU actually works and what it's doing. But, each of those billions of transistors is just a switch, so it's possible to build a CPU using any kind of electronic switch—like a relay for example.
This is actually how early computers from the '50s and '60s worked (before the introduction of transistors). Processors were built using relays or vacuum tubes. Relays were much slower, but more reliable. The performance of these early processors was based on the number of relays or vacuum tubes, and how fast they could switch. Vacuum tubes switched much more quickly than re lays, but were prone to failure.
To get a feel for this yourself, and to learn how modern processors handle arithmetic and other functions, you can purchase a Relay Trainer computer. This particular one uses 83 relays to reproduce the (very slow) operation of early computers. It's capable executing roughly 12 instructions per second; a modern CPU, for reference, executes hundreds of billions of instructions per second.
While this Single Board Relay Computer uses just relays for processing, it does take advantage of more modern semiconductors for memory and the interface. That makes it usable while still being small enough to work with. It looks like a great learning tool for those interested in how their computers actually work, and listening to the relays click and clack as they process instructions is fascinating.