Community
Recording the Visual Mind
By Appu - 7 min read
Introducing a computer vision technique designed to capture your personality – with the goal of showing you photos you‘ll love.
“The incapacity to name is a good symptom of disturbance.” ― Roland Barthes, Camera Lucida: Reflections on Photography
What about the photograph above draws your attention? Does it even draw your attention? If so, is it the expression of the mother? Is it the mood and visual path created by pose, the light, and setting of the scene, by the pure visual aesthetics? Does it recall a similar situation, or sentiment? This photo was taken by Pantea Naghavi Anaraki, winner of The Photojournalist category at The 2016 EyeEm Awards. It depicts a mother in Tehran, Iran who had to sell her possessions to pay for cancer treatment. What is undeniable here is that this photograph communicates directly to its admirers, creating both a visceral and intellectual experience that is often very hard to describe.
Nature vs. Nurture?
Your tastes are constantly evolving. For example, a photograph that appealed to you years ago suddenly loses its magic, or, conversely, you can grow fond of another one. This raises an important question: Is our appreciation of a photo a natural function of our visual system, or is it nurtured and shaped over time because of one’s exposure and experiences in life? It probably is both, but often it’s difficult to pinpoint the exact source for our emotional reactions.
However, our personal – and almost subconscious – choices are often recorded in our previous captures and curation sets. The daring question we put forward is that can we use machine learning to record our visual taste, and recreate it in a different set of photos?
A big part of EyeEm’s origin comes from its early photo exhibitions, collaborating heavily with the street pro-amateur photographers capturing authentic photography from their daily lives, mostly on mobile. This authenticity in photos is something that rubs into our curators and editors who work at EyeEm, resulting in the constant decision between our personal tastes and EyeEm’s visual aesthetic. The images on this page represent the result of these choices.
Within a few minutes, the computer generated another selection to match our “taste”:
Automatically curated by machine based on our editors’ typical choices
Putting our photo editors to the test
Visual Data Representation of our Editors’ “Taste”
The promise of machine learning is that if there is a recurrent pattern and structure in one’s data, then we can attempt to summarize them as numerical models that captures the salient characteristics and behavior of the data. This model can later be used to predict similar behavior in new, unseen data. The modern deep learning techniques don’t have an explicit model of logic, instead they try to infer a logic by trying to summarize patterns, repeatability (and also lack of it) within data as a bunch of parameters. This is collectively called the machine learning model. Appreciation of photography is strangely a bi-polar phenomenon: It is a part of experiences and literacy we gathered from the shared culture we know, or relate to, best. At the same time, it is something personal, a chance to sculpt a unique identity. EyeEm researchers first trained the algorithm with the help of expert photographers to identify a core aesthetic formed from a shared culture. Later, we build efficient, fast-to-train, interactive algorithms, that can adapt the core aesthetic models to include the personal nuances and idiosyncrasies.
Getting Results
The Week on EyeEm (TWOE) is a perfect example. Each Sunday, our team publishes a list of 20 photographs we liked from the week – out of the tens of thousands uploaded in the last seven days. We get thousands of views on these photos, which, in turn, generates thousands of likes and follows. The author of this list rotates on a weekly basis to represent a different aesthetic for each TWOE set of images. With more photos being uploaded to our platform than ever before, this might soon become an impossible task for one person, especially if we aim to be fair in giving everyone the chance for maximum exposure. So we did an exercise. Gathering data from each curator’s last six curations of TWOE, we trained our algorithm on that specific person’s curatorial preferences and asked it to create a set of images. The results allow us to have the same variety in a fraction of the time. This is the beauty of personalized aesthetics.
“It is the sculptor’s power, so often alluded to, of find the perfect for and features of a goddess, in the shapeless block of marble; and his ability to chip off all extraneous matter” – 1858, The Methodist Quarterly Review
We all strive to find subject matter and ideas that potentially appeal to our desired audiences. In some cases, the work scientists do is sculpting with data. A well-curated dataset captures the form and features, and the algorithms are the chisels and hammers that aid in carving out the fine details. The personalized component of EyeEm Vision allows photographers to gain access to the necessary tools to discover, define and share his or her artistic identity, or taste even, with a larger audience.
In 2016, we worked with the Boston Consulting Group to automate their search. All they had to do was complete a quick exercise to teach the computer what they were looking for: Presented with a grid of images, they selected the ones that matched their brand requirements. The computer then generated another selection to re ne its understanding of their taste. Within a few minutes, it created an automatic profile — and has since been altering their search results to only include the desired types of photos that fit to their brand.
Looking Forward
If photography is allowed to supplement art in some of its functions, it will soon have supplanted or corrupted it altogether… its true duty… is to be the servant of the sciences and arts — but the very humble servant, like printing or shorthand, which have neither created nor supplemented literature…” – Charles Baudelaire, On Photography, from The Salon of 1859
The fate of technology and photography has long been intertwined.
However, with any technology, it can always be a double-edged sword. It’s important to be mindful that a machine learning model does not possess any latent creativity or originality. They are modern form of journals, where we can record our thoughts and prior choices, in a scale and abstraction that was never possible before.
This feature was a joint collaboration between EyeEm’s research, content and curation teams. It first appeared in Vol. IV of EyeEm Magazine. Want to read more? Head over to swag.eyeem.com and get yourself a copy.
Header image by @NinavonAkerman