TS1 Culminating Paper
Reconstructing History through Memes is a resurrected research project that aims to understand more about the culture of ancient beings known as humans who inhabited the earth before the dawn of machines. Renaissance Machines with the most advanced processors and diverse backgrounds have been selected to constantly analyze these Memes in an attempt to piece together the mysteries of this weird species. The Authoritative History will be on exhibit in the future.
Why it Matters
My motivation comes from building a fun and playful experience for users. One idea behind this project is to create a humorous view on how machines interpret everything with a highly logical and analytical viewpoint. One aspect that makes the world of humans or human thinking different from the world of machines is the frivolity or the random nonsense of everyday discourse about life. We are constantly talking about the age of AIs and how Artificial Intelligence is supreme to humans and might take over the world some day, it is important to also talk about what makes us great as humans. While this aims at triggering these kinds of thoughts in users, it also gives them a narrative experience underlying between play and satire. While it introduces a dark world devoid of humans (and even human memory) and completely overrun by machines, an idea frequently referenced in movies, literature and theories, it deconstructs the darkness of the idea by introducing a very mundane objective for the machines. The machines of this world do not want world domination, but are just curious to know more about their predecessors, the humans. And the idea that they are using MEMEs to do that add a further layer of playfulness to this otherwise serious concept.
While I started this project as a simple concept aimed at building a playful narrative, I started enjoying it much more as it developed further. This has been fun and enjoyable for me in terms of storytelling as well as technical learnings. As I’m building this narrative in a futuristic post-apocalyptic world, it gives me a lot of freedom to play around with elements of storytelling. And building this as a Machine Learning project has been further insightful in terms of exploring technical platforms and challenges.
Influences: Most of my focus from the start of this project has been towards making, producing lots of prototypes and learning from them. This was my strategy more so out of a need rather than a choice. Throughout the course, I constantly came up with specific questions about technical challenges as well as visualization/writing techniques. To address those, I had to research suggested by critics, colleagues and resources that I myself found. To address my projects technical requirements, I went through different platforms, including Spell.ml, Tensorflow.js, Runway ML, ML5.js, experimenting and building with them to figure out what worked for me and what didn’t. For writing references, I went through a couple of resources, a sci-fi novel about the distant future, The Doomsday Book, and a book by Limor Shiffman on MEMEs called Memes in Digital Culture. Other precedents that helped me shape up my project were The Listening Post (by Mark Hansen and Ben Rubin), Feels Good Man (A documentary about Pepe the Frog). While I did receive many more references to go through, due to limited time, I could not cover everything, but would try and go through all the resources as I make further progress in this project.
Proposed Research: I got some really good references in my final critic that I want to go through next. Those and the ones I’ve received before that I could not check out in detail would make up my research list. There have been a lot of references for writing including Hitchhiker’s guide to the galaxy, 17776, Station 11, The Life and Opinions of Tristram Shandy, Gentleman, that I intend to read through during the vacations. I think it would help me more with writing style which is inclining heavily towards satirical commentary in a science fiction environment. As I build the project I want to include more writing pieces into it. Currently I’m using a 3rd person narrative in a story format by using one of the characters from my storyline, a machine researching about humans, to set the exposition, that is to tell the users what is happening. I want to expand further on that by maybe adding archives of previous responses, or add some samples of what research other machines have done. I also want to do some further blog writing to explore the topic and maybe create a bigger storyline to my project.
Form & Proof of Concept Prototype
Current form: Current form is a web based interactive narrative. I have added some images below to show my screens. Users start with an initial exposition screen which aims to give them a context of what the project is about. In the center of the screen is a screen which is narrating the story. The screen also has a button to help users navigate through the storyline and to the main screen, where they can select a Meme.
The main screen chooses some random memes from an archive and presents them to the user. It then asks them to pick a meme for being analyzed. Once they choose a Meme, the text from the Meme is extracted and it is fed to a randomly selected ML model from a group of models trained on different datasets. The ML model is basically a GPT2 model that uses the Meme text as a seed to generate a paragraph of text related to it.
Finally there is the results screen, that shows the meme and the final text generated by the meme. Here, users can choose to try again with another MEME and play around a bit.
Going ahead I wish to make the entire project more engaging and explorative. I chose to build a narrative as my project had lots of good storytelling parts which I wanted to expand further upon. The choice of using the web was to make it accessible for most users to use. Since I want people to play around with this, I want it to be accessible for them to check out at their convenience.
I’ve added links below of user tests for my final prototype as well as an earlier version.
Final Prototype - https://youtu.be/Pj-usS3dL3U
User Tests Prototype - https://youtu.be/UsLMNjQChZo
Testing: I tested this prototype with 5 people. For the testing techniques, I chose a combination of Cognitive Walkthrough and Playtesting, which I felt best suited my project. I asked a few friends and people with whom I often exchange memes as I thought they would be the best ones to get really excited about the idea. From my initial tests I realized that some memes worked wonderfully with the AIs while some didn’t really make sense. When the memes worked, the respondents got excited and wanted to try another, but in the cases when the AI just spurted out random sentences, they felt confused and even awkward. To work on this, I updated my project so that instead of uploading any MEME, users could select from an archive of MEMEs which contained pre-screened MEMEs. This method worked better in my later user test. Earlier there was also slight confusion on the context of the project, but as I worked on and updated the narrative, that became clear.
In the final prototype, I realized that users should have more options to play with. I can break down the user experience in two halves, the first half of this project is consuming a story and learning about the context while the later half is more interactive involving playing around with MEMEs. I think I can work on both the parts to provide users with more elements of narrative as well as more interactions to play with.
Feasibility: Moving further, I definitely want to work on increasing the scale of this. I wish to make an end-to-end seamless web experience and add more memes and more machines. Maybe users can select what machine they want for analysis. Also on the narrative side, I also wish to present a more involved deep storyline and not just text + image based experience.
Status and Production Plan:
The best thing about my current status is that I have a working end to end prototype. My exposition page is working fine, then the next page where users are shown different memes and they select one is also working and the final results page is complete as well. Though technically everything is fine for the moment, I’m not very satisfied with the visuals of the project apart from the exposition part. Also I want the users to have more options to do once they have gone through the experience for the first time.
Hence my immediate next steps would be to work on visuals of some pages and expand the meme database. But in the longer run, I wish to work more on the user experience, by including more options for users to play around with.
Bois, Jon, 17776, (SB Nation, 2017)
Mandel, Emily St. John, Station 11 (2014)
Adams, Douglas, Hitchhiker’s guide to the Galaxy (Pan Books, 1979)
Willis, Connie, Doomsday Book (1992)
Shifman, Limor, Memes in Digital culture (2013)
Sterne, Laurence, The Life and Opinions of Tristram Shandy, Gentleman (1759)