2. Optics & Images

You may recall from high school physics that light exhibits characteristics of both particles and waves. Different light attributes may be better thought of in terms of one or the other. For example, brightness is a function of the number of light particles — photons — arriving per unit time, whereas colour is a wave-like phenomenon, depending on the wavelength of those photons. For a given colour, increased brightness means having more photons arriving, not that the individual photons are themselves brighter.

(Note that I am being incredibly slipshod with the terminology here. As we’ll discuss in a little while, colour and brightness are not really attributes of the light at all, they are perceptual properties. But they’re a useful shorthand precisely because of that subjective familiarity.)

The energy of an individual photon depends on its wavelength or (reciprocally) frequency. Short wavelength photons have high frequencies and high energies, longer wavelengths have lower frequencies and lower energies. High energy photons can be damaging to biological tissues; lower energies can be harder to detect and broadly require larger sensory apparatus to distinguish from noise. That roughly 400 to 700 nm wavelength range of visible light represents a sort of Goldilocks zone of relatively harmless detectability in the context of human physiology. Visibility is not some physical property of light, it’s just how things are for us. Other animals will have different sensitivity ranges.

Light travels in straight lines at — famously — a constant speed. In a vacuum, that is. Matter complicates the picture in various ways. Usefully, we can think of the speed varying according to the medium through which the light is passing. Changes in medium induce speed changes and these lead to path changes. We can see this in effects such as heat haze, where fluctuating air density gives rise to wobbling. But in particular, crossing over distinct boundaries between different media — say between the air and a piece of glass — can alter the direction of the light, which we call refraction. This turns out to be super important.

If we want to use light as a medium for gathering information about the outside world, it’s helpful to know where it’s coming from. It’s not absolutely essential — there may still be some value in getting a general impression of aggregate lightness or darkness, and there are animals that do just that, but that only really scratches the surface of what we tend to think of as “vision”.

Assuming we have some kind of light detector — maybe some silver nitrate on a film or plate, a photodiode, or a nerve cell containing photosensitive pigments, whatever — if light reaches it simultaneously from different locations in the world, the information from each location will just pile up together in the detector’s response as an indistinguishable blur and we won’t be able to unpick which bit relates to what. What we would like is some form of spatial organisation, so that the light from one place all goes to the same detection point, while light from other places goes to other points.

One way to do this is to restrict the possible straight line paths of the light by forcing it all through the same point in space — a pinhole aperture. All the light arriving at any detector must be coming from exactly the direction of the pinhole because there’s nowhere else to go. This kind of setup is nice and simple and does occur in nature, especially in simpler organisms, requiring a lot less physiological machinery to evolve and build. But it has the disadvantage of capturing only a tiny fraction of the available light, discarding a lot of potentially useful information and reducing signal to noise.

An alternative is to have a larger aperture and use refraction through the curved surface boundaries of a lens to differentially bend the light passing through it, bringing the light to focus at the detector. Lens systems are harder to build than pinholes, both for human engineers and for biology, but they allow the capture of more light and more information.

The detector at a focused point has good information about one specific external position. If we have many detectors at many locations, that gives us a spatial map of optical information over a whole visual field — which is to say, an image.

Of course, focus is not perfect and not the same everywhere. Lenses will only bring to focus some parts of the external world — some range of distances from the lens. Detectors at well focused points will get strongly localised information, those where the focus is less good will receive light from a wider range of external regions and be correspondingly more imprecise, more blurry.

We are probably all familiar with this kind of locally focused imaging from using cameras, which combine a lens system for light gathering and focusing with some kind of spatial array of detectors — like a film frame or CCD — to perform the actual image capture.

Human vision is quite unlike photography in various important ways, some of which we’ll get into. But there are also enough structural similarities in the early stages that cameras are at least analogically useful as a model for starting to think about eyes.

The Vision Thing

[To bully myself into actually writing the lecture on visual perception I’m due to deliver in January, I’ve decided to try drafting the wretched thing as a series of blog posts. This is probably a very bad idea. It’s a two hour lecture, so there could potentially be quite a few of these posts, but also the chance of bailing is pretty high. Whatevs. Here goes nothin’.]

1. Preamble

To start with the bloomin’ obvious: vision is the process — the collection of processes — by which we gather information about the external world through the medium of light. To be precise, like Thomson and Thompson, that’s the medium of visible light — a relatively narrow band of electromagnetic radiation of wavelengths roughly 400 to 700 nm.

Light is a pretty great medium for gathering information — almost the definitive one. It propagates really fast over long distances, even through a vacuum. It travels, at least to a reasonable approximation, in straight lines. It interacts with matter and participates in chemical reactions, making it possible for biology to come up with mechanisms for detecting it. And it’s plentiful. There’s a lot of it about, at least during the day, thanks to an enormous nuclear furnace burning in the sky.

Vision is so useful that it has evolved numerous times over the history of life on Earth, with a fair bit of variation in the implementation details. There are a bunch of different structures and configurations of optical sense organs — the eyes of a fly or a flatworm or a scallop are quite different from our own — although some of the underlying biochemical components are pretty well conserved. We’ll touch on those in a little while.

Vision is also so useful that it’s a bit overpowering. It’s the dominant sense for most humans, one on which we rely heavily, one that radically shapes our understanding of the world, both literally and metaphorically.

The vocabulary of vision pervades our language, encompassing much more than mere optical detection: we say “oh, I see” meaning “I understand“, or “let’s see” meaning “we want to find out” or “we’ll see about that” meaning “I will act to prevent it.” The word vision doesn’t just refer to the sense of sight, but extends to intellectual brilliance, to clarity and drive, to divine revelation. We describe a great creator as a visionary, someone who predicts the future as a seer, or a clairvoyant — literally one who sees clearly.

Much of human culture revolves around the visual — painting and drawing, photography and movies, games. Practical necessities like food and clothing are wrapped up in visual aesthetics. The written word is primarily mediated — read — through our eyes. We have all manner of advanced technology dedicated to the enhancement of vision — microscopes and cameras and telescopes — and to the production of visual stimuli — from paper and pencils to printing and screens to HUDs and holograms and VR headsets. It is one of the major ways we interact with computers and software, often the main component of a user interface.

At the same time, vision is complex and fragile and lots of things can go wrong with it. Visual impairments are extremely common. People may have difficulty with focus or resolution close up or far away or — especially when you get to be ancient like me — both. People — particularly men — may be unable to distinguish some or all variations in colour. People may be unable to see in dim or bright conditions. They may lose parts or all of their visual field to obstructions in the eyes or degeneration of the sensory tissues or loss or damage to perceptual pathways. Some of these problems can be readily mitigated with technology — very many of us wear glasses or contact lenses, whether all the time or for specific tasks — other problems not so much.

So, given both vision’s centrality to human endeavours and its frequent failings, it is important to understand how it works and how it goes wrong, and try to find ways we can make the most of it while also maximising accessibility.

Qu’est-ce que c’est?

Occasionally is right. Yeesh.

Predictably—and indeed predicted—this happened, though currently in a pretty nascent state. Most of the notebook for Lab 2 is now there. Lab 1 is mostly done but not yet saved to GitHub. Lab 3 is also not on GH, and exists only in a very sketchy form, but contains stuff like this:

This sort of nonsense probably wouldn’t surprise anyone who has Nebulized.

Killer

That word apace in my previous post might have been a bit of an exaggeration. A snail’s pace, perhaps. There is progress, but not as much or as fast as I’d like. Things are creeping along.

The psychophysics and perception lab stuff now has a home on GitHub. I seem to be becoming a collector of GitHub organisations named after UCL module codes. An obscure hobby, admittedly, but mine own. I suspect there will be more to come—probably at least comp0161, though to date Colab alone has sufficed for that. My generative music sketch from the summer has decomposed into two parts (lab1 & lab2, both still very much works in progress), with a third chapter on synthesis and effects currently just a twinkle in its father’s eye. These labs are a bit short (one UCL hour) and their schedule a bit random, so the narrative arc might not really hang together. The arc for comp0160 ought to be more solid—weekly two hour practicals for a month, with a high value season finale (40% of the marks)—but I’m still shaky on many of the details. Especially how it all comes together at the end. I hope I won’t have to bring in a man with a gun.

All this has been somewhat overshadowed by sudden parental death. Not my own parent, but the next best thing. We’re at an age where such things should perhaps not come as a surprising—the mother of some of Ian’s friends from the same village died just a couple of days later—but of course they always are surprising, it always comes as a shock.

So, yes, we are all shocked.

I am back in London for the moment, but will be up to Wales again for the funeral (strictly, the memorial) next week. It’s sure to be a rib-tickler. In the meantime: students and coursework and panic and perhaps even—very, very occasionally—blogging.

Psycho

Development of teaching materials for the new academic year proceeds apace. Currently at the top of the agenda for no very good reason: lab exercises for COMP0160 Perception & Interfaces. Here, have a prototype.

This is intended to be just one small part of one session, but it’s a starting point. Also a moving target—next week it will probably have changed.