I Want to See Movies of My Dreams
The heat of the mid day sun is curtailed by a ceiling of gray clouds, but with the clouds is the promise of rain and with that, the kiss of wind. It pushes down on the cyclists as they wind their way through the layers of clay walls that rise around the river, cut by millions of years of erosion. Yet the clouds offer some respite from the heat and as they struggle to keep themselves upright the promise of precipitation finally resolves and a not unpleasant mist ensconces them. The rain enhances the perfume of the prairie: the smell of sage and the faint hints of manure. In this heady mix the lead rider becomes unstuck in time.
He is fifteen and watching the Angus mill about a pasture, astride the red quad, idling away an afternoon while dreaming of outlaw cowboys and an imagined west culled from film and dozens of Louis L’Amour novels. He watches the dark cobalt of an impending storm and in the pre-rain stillness he hears a far off voice just as lightning begins to arc across the sky. “Dinner!”
He is thirty-seven and contemplating throwing his bicycle into a field and waiting for a respite from the wind.
He is fifteen and slathering his grandmother’s freshly baked rolls with yellow slabs of margarine and Saskatoon jam.
He is thirty-seven and the spray of rain is occluding his vision.
Close your eyes for a moment and recall a childhood memory. Something encompassing. The sort of memory that “feels like only yesterday.”
Smell, sight, sounds. Some moments, an aspect of a memory, can seem so clear that the temporal distance from now to that moment seems to dissolve. In our ever media saturated world, there is an instinct to view these memories as a library of film clips and, of course, said medium has also popularized the notion of eidetic memory (note, "eidetic" and "photographic" are used interchangeably, though they are not quite the same... but the former sounds more "sciencey" so let's go with that). In fact, this notion that our brains store memories much like files has led to all manner of literary interpretations. Stephen King employed this concept as a “memory warehouse” in his book (and the subsequent film) “Dreamcatcher.” (It's neither a good film nor book, FWIW.)
However, this notion is folly.
To gain better insight into how long term memory is stored, consider dreams.
Dreams, which occur in the final phase of sleep, are usually quite short:
"Dreams themselves usually last between a few seconds to 30 minutes in length.” [1]
However, the sensation of time distortion in dreams is altogether common. There is evidence that in lucid dreaming (e.g. dreaming wherein the subject is aware of the fact that they are dreaming), motor actions are near-real time[2] if not longer. Yet, this methodology (which involves counting and tracking of REM), is stilted in that lucid dreams are not the norm. But dreaming does afford the mind the ability to compress sensations. Studies have indicated that dreams which appear to have a consistent narrative [3] are actually composed of disconnected events wherein the subject interpolates the “missing frames” into a more cohesive story. We could, of course, attempt to suss out what this means or if this is an effect of implicit training due to subject’s exposure to things like films and books. But let’s put a pin in that notion and get back to memories.
Memories are not like film clips. First, there is the obvious distinction that our memories encapsulate more information than a film. This is not to say that the resolution, for example, of an image in memory is superior to that of the medium of film (e.g. “better picture quality”) but that more and other information is captured. Smells, sounds, physical sensations… If I ask you to recall the feeling of sandpaper against your skin, or the sensation of sand in your shoes, you can do so easily (perhaps more so as both of these sensations are unpleasant) and probably even recall a particular time and place where you remember experiencing those sensations. So there’s the obvious factors in which a memory can be different than a film. Then there is a litany of semantic and contextual data that is also included in your memories.
Imagination, as both a capability of mammals and a trait particularly employed by Homo sapiens in particular, does much work to “fill in the blanks.” I remember well the smell of my grandmother’s rolls but in the aforementioned snippet of memory, do I really remember that particular instant? Or do I remember a collection of experiences and employ that knowledge in that memory.
I am not, of course, a neuroscientist and my knowledge in the fields of both neuroscience and psychology are limited to some undergraduate courses and books I’ve read, but this all touches on something that is worth considering: context is important in recall.
When I smell sage, no matter where I am physically, I recall early mornings on the Alberta prairie. I can recall the way the dew glistened on the grass and the faint smell of cows, even the wool work shirt I often wore, that never quite fit right and braced against my skin. Now, I am intelligent enough to realize that aspects of these memories are perhaps distorted. I could try to do something like count the cattle that is a part of these "memories" (I want to type "images" but they aren't really images), but I suspect that would be more a trick of imagination than anything with any veracity.
Herein lies a corollary: your memories are often flawed.
The way in which autobiographical memories are recalled (and I will forgo delving into the common consensus on long term memory, for the sake of brevity and simply because I am only an arm-chair "expert") is an amalgamation of archived bits and pieces and interpolation due to imagination [4]. It is this aspect that allows us to fully distort memories and part of what makes our memories somewhat dangerous. There is this notion that because memories are recreations versus something akin to rewatching a film clip, they are especially susceptible to distortion, e.g. what has now been called “The Mandela Effect” isn’t new [5]. There is an apocryphal tale about a researcher who remembered his uncle’s Jeep being a bright yellow, when reality (and photographic evidence) show that it was red. In fact, despite what you may believe, because memories are more about a collection of data points rather than a perfect encapsulation of a particular event, you are quite likely to “get the details wrong”[6].
Now, this is a blog about a project, particularly about a problem space that encapsulates various aspects of Computer Science from Human Computer Interaction (HCI) to Neural Networks (NN), Computational Linguistics (CL) and other "sexy" buzzwords and not meant to be an exhaustive look into the human memory and this is not an attempt to replicate the way in which memory is stored in the human mind into some type of NN… that would be utter folly and isn’t at all wherein such applications are useful. Rather, I posit this notion of how our memories work to solve an entirely different problem: searching.
If you have read my previous post regarding my over-arching interest in what I can only call “the thoughtb.us experiment” I am intrigued by this notion that text based searching places a limit on the user and requires the ability to leverage higher order concepts to mere text. Basically, because we engage in search actions by limited filters (text, date, document type), we are not properly employing all available resources.
If a system existed such that it had access to, at least, all electronic artifacts (Google Docs, email, text messaging, Slack channels, and on) and information germane to those artifacts (time, time frame, content, even sentiment), the loss of information due to the inability to search effectively could be reduced.
In research interviews conducted at my previous job around this problem (herein lies the problem with trying to say anything with any scientific veracity: all that work was scrapped in acquisition and I can't share it), we found that due to the heterogeneous nature of various tools employed in documenting everything from law cases to long term sales pitches, users often lost important or relevant information because retrieval was difficult. A lawyer working for the District Attorney in Boulder County, for example, had notes in Word documents, emails, and PDFs (not to mention completely untracked physical files) often lost important details or missed aspects of a client’s case solely due to both the workload (never assume that an attorney doesn’t work hard) and the disconnected way in which the documents themselves were stored. Police officers were likely to submit scans of written reports (and OCR is still not sophisticated enough to fully convert scrawled text from one subject into actionable data) via one system while the DA’s office used an entirely different means of storing said information.
Now, I cannot pretend that some of these difficulties can be solved with a given approach to visualizing search or even changing search parameters or the paradigm of search. Simply getting all relevant information into a digital system is another hurdle beyond my meager means. However, little published research actually looks at changing how search is approached. There is much work on optimizing results based on things like keywords [7], but much of this work is done behind closed doors in proprietary systems and little, if anything, has changed in how users are enabled to search.
What if, instead, a user could search for a document by remembering a different set of criteria:
A time frame.
- Related events (e.g. an upcoming party or the termination of a coworker’s employment).
- Related documents (remembering a particular task, or, for example that your development team had just hit a release date encapsulating a set of features).
Moreover, remember what a user is actually doing (you, in this case): looking for information. A document/artifact (and I use “artifact” to encapsulate things not strictly considered documents… like a Slack discussion or even a git commit) is a source of information, a set of ideas or data around something.
In many organizations, information is lost merely because it is hard to find. The easiest example of these problems are when a decision is made, at some point, and that decision turns out to be incorrect. The vendor providing the widget ran into delays or an API proved to be problematic. It is the nature of a modern organization, employing workflow analysis via something like Kanban or the like, to introspect and attempt to determine future courses of action from this type of data. And, even more tactically, a sales person might care about whether or not a relationship was soured due to some other event they cannot quite recall.
Now, many of these things are done in a rather manual fashion. We sign contracts, we spend time making pointed artifacts (e.g. meeting minutes or the like), but these are imperfect representations of the information at hand. In yet another set of research interviews, we found that users who did not lose information as easily did so by being extremely pointed in their own documentation: taking careful notes, organizing artifacts, and the like. However, even for these users information was still lost and, more often than not, these artifacts were never generated.
From an economic standpoint, it is difficult to determine how much of a cost this truly is (in terms of both utility and wasted time, which is to say money). It can be posited that such a cost exists, however the severity of the issue and the ROI of a “better” approach to search might not make financial sense as the investment in such a speculative problem space is rather costly.
But, at the end of the day, it is an interesting exercise.
To that end, I’ve begun work on a sort of loose interface that can tie into a larger system. As it is an interesting project, involving many aspects both mundane and sublime, I am pressing forward. But if you’ve made it this far, dear reader, I’d like to elicit your feedback. Is this a problem you’ve encountered? What difficulties do you have in retrieving information? When are you pressed to find information in an artifact?
If you have a moment, feel free to fill out this short survey (I have no incentives for you... other than maybe I will write another lengthy blog post and, uh, science!)
(And yes, the title of this post is a reference to a song and I apologize for the "borrowing" of the Vonnegut concept of "unstuck" in time, from Slaughterhouse Five)
[1] The Psychology of Dreams: Inside the Dream Mind. http://online.brescia.edu/psychology-news/psychology-of-dreams/
[2] Erlacher, D., Schädlich, M., Stumbrys, T., & Schredl, M. (2014). Time for actions in lucid dreams: effects of task modality, length, and complexity. Frontiers in Psychology, 4.
[3] Pace-Schott, E. (2013). Dreaming as a story-telling instinct. Frontiers in Psychology, 4.
[4] Bluck, S. (2009). False memories: What the hell are they for?. Applied Cognitive Psychology, 23(8), 1105-1121.
[5] Collective False Memories: What's Behind the 'Mandela Effect'? - The Crux http://blogs.discovermagazine.com/crux/2017/02/16/mandela-effect-false-memories/
[6] Strange, D., Hayne, H., & Garry, M. (2008). A photo, a suggestion, a false memory. Applied Cognitive Psychology, 22(5), 587-603.
[7] Ingale, Dhanashri & Kshirsagar, M. M. (2016). A review on an approach to infer user search goals for optimize result. IEEE: Futuristic Trends in Research and Innovation for Social Welfare (Startup Conclave), World Conference on.