Emotions are data

Emotions are the essence of humanity and a fascinating material that can also be transferred into data and then processed.

How do we recognize emotions? How do we know if someone is experiencing joy or sadness, if they are scared or surprised? We learn this by living among people and experiencing different emotional states ourselves. Machines can record these states because people send infinite amount of information through facial expressions, posture, gestures, breathing speed, voice tone and heart rate. Ways of expressing emotions are culturally conditioned and thus they differ across the world. They also depend on gender, personality and temperament. All this information can and does become a valuable material, which is then recorded and analyzed. Ivy Go offers a web app that records data in the form of facial checkpoints, then processes them and presents a visual record of emotions, showing their different degrees of intensity. Haelo Design captures love gazes and turns them into a data collection, i.e. hundreds of thousands of points, and then transforms them into a physical object that illustrates how lovers look at each other. 
Designers work on numerous solutions that are based on emotions. One of them is Empath—an application using artificial intelligence, which can recognize emotions based on human voice—in real time, regardless of the language; it analyzes the pitch, tone, speed and volume. Currently, it recognizes only four emotions: joy, peace, anger and sadness (www.webempath.com). One of the most beautiful projects in recent years that studied the relationship between emotions and data materialization is “Dear Data” by Giorgii Lupi and Stefanie Posavec. Every day the artists recorded information describing selected aspects of their lives and drew them on postcards. Next, they sent the designed postcards to each other by post. The result was a collection of handwritten sets of tangible data in the form of hundreds of postcards, full of beautiful emotional visualizations. To see this project, please visit: http://www.dear-data.com. 
Varvara & Mar, an artist duo from Estonia, monitored the work of their hearts for a year. They recorded one of the key elements of human physiology, which is a component of the experienced emotions. They say in their commentary that today the algorithms manage to guess our tastes, preferences and emotions: “we have become as predictable and programmable as never before.”








Emotions are also a data. In our brain, the fact that we are happy, sad or angry are just nerve impulses which take the form of information.

prof. dr hab. inż. Ryszard Tadeusiewicz, AGH w Krakowie








2015

Dear Data

Giorgia Lupi, USA
Stefanie Posavec, UK


Drawing data enables us to connect with ourselves at a deeper level. Sharing such drawings – developing a friendship.

Dear Data is analog data drawing project consisted of 104 postcards. Each week for one year, Lupi and Posavec gathered information on chosen aspects of their lives, e.g., moments of expressing dissatisfaction or annoyance about a particular thing, or positive feelings they were experiencing during the past seven days. They each visualized the collected personal data on postcards and sent them to each other across the Atlantic – Lupi from New York, Stefanie from London.
On the front of the postcard, there is a unique representation of weekly data, on the other side, detailed explanation of how to read the drawing. Dear Data documents a process of developing a friendship through data visualization. 
Instead of using data to become more efficient, Lupi and Posavec argue we can use data to become more human and to connect with ourselves and others at a deeper level. 
Now Dear Data project is in the permanent collection of the Museum of Modern Art’s in New York.
The presented set of postcards represents Week 7: A week of complaints.

Through the course of our year of Dear Data we appreciated that by taking the time to stop, notice, and collect data on our emotions we were ultimately able to know both ourselves (and each other!) on a deeper level. Data collection and drawing is how we make sense of ourselves and our world!

Stefanie Posavec

Emotions, just like data, are characterized by beautiful complexity. A deeper relationship between them is revealed when we start expressing emotions numerically. However, I believe that this relationship should be monitored more closely for possible abuse of power and financial exploitation. We are obsessed with transforming all emotional experiences into measurable, standardized and marketable opportunities. Instead, we should accept all shades of human emotion, or there is a risk that technology will become the custodian of our emotions.

Ivy Go

2019

Collector of Emotions

Ivy Go, Holandia


People feed Internet systems with their emotions, and this emotional content is turned into data. Systems transform emotions into data because they have been pre-programmed to do so by people, who have also defined the purpose of these processes.

Collector of Emotions is a web app that tracks and stores the full spectrum of human emotions. The data from facial expression readings are translated into different colors, in contrast with the numerical approach used in regular emotion tracking systems. The concept of a tracking system that collects a wide range of non-standardized emotions was formulated by a designer. Then, the system was developed programmers: a built-in computer webcam records facial expressions and facial checkpoints. Next, the data are processed and presented as a visual record of emotions, including their degree and depth. The application encourages us to rethink allowing Internet systems mining the data concerning our emotions. These systems are primarily designed to bring profit, not broaden human experience. The designer poses the question: how do you feel when a system deconstructs, analyzes and tracks your emotions? The application is available at: www.emotion.fyi.

2014

Manifesting the Look of Love

Haelo Design, USA


Love gazes are data that convey beauty. You just have to capture (!) and save them, freezing in time a unique and elusive moment.

The look of love is one of the most personal, palpable, and sought after human sensations. Manifesting the Look of Love is a proposed design service that creates highly personalized objects designed by this intimate form of communication between two lovers. The service allows couples to mark significant intervals in their relationship with the creation of a completely unique artifact, directly formed by how they see one another. The gaze of each couple is mapped using custom eye-tracking technology — it captures the gazes of those in love and turns them into a set of data, i.e. hundreds of thousands of points, informing where each person’s eyesight was directed, where they focused it and for how long. The resulting data is transformed into a physical object through parametric modeling tools and digital fabrication processes. These objects are realized in the material most closely associated with the duration of the couple’s relationship at their anniversary. A lifetime subscription model provides couples with an annual appointment to collect new gaze data and evolve their bespoke collection of highly symbolic designed objects.


2019

Listening Cups

Audrey Desjardins, Timea Tihanyi, USA


Sounds can be fun, irritating, relaxing or annoying. What we feel is inextricably linked to the outside stimuli. The recording and materialization of sounds preserves the feelings which they evoked.

The ListeningCups is a set of 3D printed porcelain cups embedded with datasets of everyday ambient sounds. This project emerged from a collaboration between a ceramic artist Timea Tihanyi from Slip Rabbit digital ceramic lab and an interaction design researcher Audrey Desjardins who were exploring meaning-making around everyday data. Data of home noise levels were used to drive the texture on the surface of the 3D printed cups. Bumps on the cup represent short snippets of sounds recorded in various everyday places: Desjardins’s home, a restaurant, Slip Rabbit Studio, or street. The joint work of Desjardins and Tihanyi consisted in developing a process of recording ambient sound data, namely recording volume levels (decibels) and converting this information into machine code information (G-code), which then produced a texture on a Potterbot7 ceramic 3D printer.

By giving a physical shape to the data collected over time in the form of a familiar and highly tactile functional object, we hope to spur imagination, memory, and reflection, as fingers cross a specific bump or a smoother area. Bringing back this tactile data to environments of daily life—from which the data was collected in the first place—allows the user to live with the new material object and the data themselves. This tactile presence also allows for new moments of storytelling around data, stories of past moments, but also questions and imagined events that may or may not have happened. There is the potential to create a space to engage with data differently, through curiosity, emotion, and investigative practices. By materializing data, this project also highlights ‘data accidents’ that occurred during data collection, data transcoding as well as material fabrication, showing how data are not perfect, rather they are open to subjectivity and aesthetic sensibilities.

Audrey Desjardins

In a monitoring and data-based society, the algorithms have the ability to measure our likes, preferences, connections as well as emotions and relations. We have become predictable and programmable like never before. We give away all our personal data for free, which are then used for profiling and forecasting. According to Christian Fuchs, even if people enjoy it, it is still a form of exploitation.

Varvara & Mar

2017

The Rhythm of Heart

Varvara & Mar, Estonia


The heart reveals emotions, its rhythm naturally synchronizes with what we are experiencing. Our heartbeat is beyond our control, and that is why it is a true reflection of our feelings.

Varvara & Mar artists from Estonia monitored the function of their hearts for a year. They translated the heart rate monitoring data from FitBit bands into the rhythm of metronomes. The audience of the installation can listen to the beat of the artists’ hearts, and even set the metronomes to a selected date and time to recreate the rhythm from a chosen moment from the artists’ archives. What you hear is related to the emotions that the artists were experiencing at a given moment. It is a critical project, encouraging reflection on the desirability of data mining by third parties and draws our attention to the monetization of data in the times of capitalist surveillance. The artists also initiate a discussion on the meaning and changing form of personal archives.