Sunday, November 06, 2016

Nostalgorithms

Nostalgia is a feeling, to start with. We have songs and photographs we happen upon that conjure nostalgia. We have articulations of nostalgia, in poetry, in the lyrics of songs, in films, TV shows, novels. And now, we have algorithms that serve up content (songs, photographs, news) that make us feel nostalgic.

The “Your memories on Facebook” function fascinates me, as a potential mechanism for conjuring nostalgia. It's hard to know precisely how the algorithm works - how and when it decides to bring a photo up from the past and ask you whether or not you'd like to share this photo again on your timeline - but it appears to bring up pictures from the same day of the year in previous years, most likely at least 2 years from the past. It also is likely that photos/memories are chosen based on the amount of "likes" or comments they received at the time. 

That’s certainly one of the simplest approaches, and it works well enough, but perhaps not as well as it could work. I’ve only really heard people talk about this aspect of the Facebook experience (as is the case with many aspects of any kind of technology) when it doesn’t work. People make note of the times when Facebook served up a picture on an ex or, worse, a deceased love one. It’s clear that it doesn’t work perfectly, and yet it works well enough to persist. 

Does that algorithm learn from the times it presents unpleasant memories to users? Probably. Perhaps it starts by serving up memories, allowing a certain period for the memories to “steep," and, after a period of trial and error, it would be possible to identify certain types of memories that people elected to share. These types would be defined by objective qualities the shared memories had in common, qualities that set them apart from the non-shared memories. The algorithm is “dumb” in the sense that it doesn’t know anything about the concept of nostalgia, or the individual users’ lives, or about human emotion in general. But if you give it enough data, enough pictures, enough memories, it will probably get better at serving up pictures that you want to share, pictures that tap into something that we would call nostalgia. It learns not to serve up those pictures of your ex.

Perhaps there's an unseen pattern or signature to nostalgia that could be revealed by the algorithm. It's not just a matter of how much time has passed that makes us nostalgic for something. It has to do with the specific contours of social relations and feelings, all of which leave an imperfect imprint in our social media archives (less and less imperfect as more and more of our social/emotion lives are channeled through social media). 

Here's an example pattern using data that a social media company like Facebook could collect: Optimal nostalgia resides in the pictures with that person you appeared with in other pictures and exchanged frequent IMs with for a period of three years after which there were fewer and fewer pictures of the two of you together and fewer IMs until the trail went cold, but you were still "liking" and occasionally commenting on their posts, though this wasn't reciprocated, suggesting a kind of unreciprocated longing for re-connection. Or maybe it takes into account the time of day at which it was posted (maybe people are more nostalgic about things that happened at night) or the place (maybe nostalgia clings to certain places more than others, or it requires a certain physical distance from our current locations, at least 1,000 miles). Maybe it's all there, residing in the metadata. 

I think about nostalgia in terms of music, too. Pop music (and movies/TV shows that use pop music) have worked with a crude version of the nostalgia principle for decades, if not centuries. Artists arrange a song in a familiar way, or include a certain familiar phrase or melody, so as to strike a particular emotional chord in the listener. Genres are revived in part out of nostalgia. But algorithms could give us something much more fine-grained, more personalized. Imagine that your entire music listening history was archived (as will be the case for people starting to listen to music in the age of streaming services like Spotify, Pandora, or YouTube). The program would know that you really loved a particular song (you played it 100 times that one week in 2010) but then seem to have forgotten about it (you haven't played it since). One of life's great pleasures is hearing that song you loved but have not heard in years. Part of you knows the rhythm and the lyrics, but another part of you has forgotten them. Your ability to sing along with the first verse feels instinctual, but you can't remember exactly what the chorus was until it comes crashing in, and you think, "how could I have forgotten this?" 

Maybe the music program is integrated with your preferred social media app. The social media app has a rough indication of your mood and what's going on in your life. It can make a pretty good guess as to when you're ready for an upbeat song and when you're ready for something more introspective. Maybe it knows that you found a love song when you weren't in love and seemed to like it but couldn't listen to it too frequently because you weren't in love. And now that it knows you're in love, you're ready to hear it again. Maybe it knows that the lyrics to another song will be more poignant to you now that you're 40. 

There is a visceral revulsion at technology colonizing overly-personal or artistic realms of human experience. All fine and well if the algorithms make shopping more efficient, but nostalgia? Memories, and our experiences of them, are tied to identity. This may account for the way in which we need nostalgia triggers to feel serendipitous. The idea of an algorithm writing poetry is a bit unsettling, but what about an algorithm that can conjure the feeling that inspires poets in the first place?