Living Pictures

Inspiration

Being a team of three international students, we shared a similar emotional requirement to express nostalgic feelings.

We chose to augment a photo frame because it has always been an object that stimulates emotions and is associated with memories and nostalgia. It also felt like a suitable tangible medium to connect people with their loved ones whose pictures are already placed in a traditional photo frame.

Understanding Social Interactions

We tried to understand the nuances of social interactions that we have with the people that are closest to us. Even without explicitly verbalizing our feelings and saying out loud that we love them, it is the simplest of gestures and shared moments that convey the feelings.

"Staying in touch" and "sharing a moment" became keywords for our interaction and we wanted to create something that would instigate such feelings by interacting with the objects in the immediate environment.

Sharing a moment together imitates that magical feeling of being in the other person's presence without having any urgency and need to say something, the feeling of knowing that they are there, the feeling of living in the moment together, of being alone together.

Interaction Ideation : Red+Blue = Purple

Seema

Staying in Touch

Connecting with your loved ones should not be a task that you accomplish only after dodging obstacles and working through interfaces. It should be simple and feel natural, something that we would do anyway, as simple as holding a picture frame in hour hand when you miss someone.

We used capacitive touch sensors and installed them around the frame, so when a user lifts the frame, it can detect the touch and connect them without making the users go through layers of interfaces

Testing out the Capacitive Touch

Circuit Diagram for assembly

Satyan

Living Pictures Code

// This #include statement was automatically added by the Particle IDE.
#include <neopixel.h>

#include <math.h>
// NEOPIXEL VARIABLES
#define PIXEL_COUNT 4
#define PIXEL_PIN D7
#define PIXEL_TYPE WS2812B
Adafruit_NeoPixel strip(PIXEL_COUNT, PIXEL_PIN, PIXEL_TYPE);
// INPUT VARIABLES
int inputPins[] = {D2,D3,D4,D5}; // TOP, RIGHT, BOTTOM, LEFT
bool inputState[] = {false,false,false,false};
// EVENT VARIABLES
long lastPublishedAt = 0;
int publishAfter = 100;
String eventRoot = "diot/2019/css/livingpictures/";
// LED FUNCTIONS WILL MAKE GOOD USE OF THESE 
bool home[] = {false,false,false,false};
long hTime[] = {millis(),millis(),millis(),millis()};
bool away[] = {false,false,false,false};
long aTime[] = {millis(),millis(),millis(),millis()};
int red[] = {0,0,0,0};
int blue[] = {0,0,0,0};
int green[] = {0,0,0,0};
int frames = 4;
// ADSR VARIABLES
double attackLevel = 1.0;
double sustainLevel = 0.5;
double vibratoRate = 1000.0;
long attackTime = 3000;
long decayTime = 5000;
long releaseTime = 10000;
//ENDED UP NOT USING THE ADSR FUNCTION
double ADSR(int clr, bool press, double pressTime) {
    double v = 0.5;
    if(press) {
        // ATTACK
        if((millis() - pressTime) < attackTime) {
            v = double(millis() - pressTime) / double(attackTime);
            v = 1 - ((1 - v) * (1-v));
            v = (0.0 * v) + (attackLevel * (1-v));
        } else {
        //SUSTAIN
        v = 0.5 + double(sustainLevel * sin(float(millis()/vibratoRate)));}
    } else
    //RELEASE
    {
    v = (clr / 255.0) - (1.0/double(releaseTime));
    if (v < 0.0) {
    v = 0.0;
            }
    }
    return v;
}
void setup()
{   
   // I/O PINS SETUP
   pinMode(PIXEL_PIN, OUTPUT);
   for(int i = 0; i < frames; i++) {
    pinMode(inputPins[i], INPUT); 
   }
   // NEOPIXEL INITIALIZE
   strip.begin();
   for(int i = 0; i < frames; i++) {
   setRGBColor(i,0,255,0);    
   }
   strip.show();
   strip.setBrightness(255);
   // HELPS CAPACITIVE TOUCH SENSOR SETTLE
   delay(2000); 
   for(int i = 0; i < frames; i++) {
   setRGBColor(i,0,0,0);    
   }
   strip.show();
   // EVENT SUBSCRIPTION
   Particle.subscribe(eventRoot, handleSharedEvent );
}
void loop()
{   
    //Checks Strips for touch, publishes event if touch detected or if touch no longer exists
    for(int i=0;i<frames;i++){
    int buttonState = digitalRead( inputPins[i] );
    if( buttonState == LOW)
    {
        if(!inputState[i]) {
            //FIRST TOUCH DETECTED
            inputState[i] = true;
            String j = String(i);
            publishMyEvent( j + "N");
        } 
    }else{
        if(inputState[i]) {
            //TOUCH FINALIZED 
            inputState[i] = false;
            String j = String(i);
            publishMyEvent( j + "F");
        }
    }
    delay(5);
    }
    //Update lights depending on status
    for(int i=0;i<frames;i++){
        int redColor = red[i];
        int blueColor = blue[i];
        int greenColor = green[i];
        if(home[i] && away[i]) {
            //Both sides are touching the same side of the frame
            blueColor = 255;
            redColor = 255;
            float val = (exp(sin(millis()/2000.0*M_PI)) - 0.36787944)*108.0;
            redColor = int(val);
            blueColor = int(val);
            } else{
        if(away[i]) {
            redColor = 255;
            } else { redColor = int(redColor * 0.9);}
        if(home[i]) {
            blueColor = 255;
            } else { blueColor = int(blueColor * 0.9);}    
        }
        red[i] = redColor;
        blue[i] = blueColor;
        green[i] = greenColor;
        uint32_t c = strip.Color(redColor, 0, blueColor);
        strip.setPixelColor(i, c);   
        strip.show();
    }    
    
}
      /* if((millis()-pressTime) > 7000) 
            {
            setRGBColor(0,0,0);
            strip.setBrightness(255);
            } 
            
        if(millis()-updateTime > 1000 && (millis()-pressTime) < 7000){
            updateTime = millis();
            int nw = int(brightness * 0.6);
            Particle.publish( "Brightness", "Seema" );
            strip.setBrightness(nw);
            strip.show(); */
void setRGBColor(int p, int r, int g, int b ){
  strip.setPixelColor(p, r, g, b);
}
void publishMyEvent(String data)
{
  // Remember that a device can publish at rate of about 1 event/sec,
  // with bursts of up to 4 allowed in 1 second.
  // Back to back burst of 4 messages will take 4 seconds to recover.
  // So we want to limit the amount of publish events that happen.
  // check that it's been 10 secondds since our last publish
  if( lastPublishedAt + publishAfter < millis() )
  {
      // Remember our subscribe is matching  "db2018/paired/"
      // We'll append the device id to get more specific
      // about where the event came from
      // System.deviceID() provides an easy way to extract the device
      // ID of your device. It returns a String object of the device ID,
      // which is used to identify your device.
      String eventName = eventRoot + System.deviceID();
      // now we have something like "diot/2019/paired/0123456789abcdef"
      // and that corresponds to this devices info
      // then we share it out
      Particle.publish( eventName, data );
      // And this will get shared out to all devices using this code
      // we just pubished so capture this.
      lastPublishedAt = millis();
  }
}
// MESSAGE DATA PARSER FUNCTIONS
int parseMessageIndex(String data){
    if(data.indexOf("0") != -1) {
        return 0;
    }
    if(data.indexOf("1") != -1) {
        return 1;
    }
    if(data.indexOf("2") != -1) {
        return 2;
    }
    if(data.indexOf("3") != -1) {
        return 3;
    }
    return 4; // Means something is wrong
}
bool parseMessageStatus(String data){
    if(data.indexOf("N") != -1) {
        return true;
    }
    if(data.indexOf("F") != -1) {
        return false;
    }
    return false;
}
// SETTERS FOR HOME & AWAY ARRAYS
void setHomeStatus(int i, bool s){
    if(s) {
    hTime[i] = millis();        
    }
    home[i] = s;
}
void setAwayStatus(int i, bool s){
    if(s) {
    aTime[i] = millis();    
    }
    away[i] = s;
}
void handleSharedEvent(const char *event, const char *data)
{
    String eventName = String( event ); // convert to a string object
    String eventData = String( data );  // convert to a string object
    String deviceID = System.deviceID();
    // Parse message to get necessary information
    int index = parseMessageIndex(eventData);
    bool status = parseMessageStatus(eventData);
    
    if( eventName.indexOf( deviceID ) != -1 ){
    //Change the information on corresponding pixel
    setHomeStatus(index,status);
    return;
    }   
    //Change the information on corresponding pixel
    setAwayStatus(index,status);
}

Cem Click to Expand

How Living Pictures works

Seema

Feedback

"It's profound and clever to use a picture frame to convey emotion because it is cheap and accessible. The decaying of light and allowing the other user to catch the moment in time to respond adds a layer of intimate connection and provokes a desire to interact at the same time without explicitly stating it." - Eddy, School of Architecture, Director of Computational Design at CMU.

"What if the other person is sleeping and miss out on your message, was that intentional to not send push notifications? " - Robert, iDeaTe Lab at CMU

"Allowing users to set preferences of delays, gestures and responses was an important piece in your interaction. " - Professor Daragh, Designing for IoT

Future interaction

We think about why would someone want to use it for a prolonged period, would they outgrow it? Therefore, we propose to add a level of abstraction to our interaction design leaving it to the users to define their own way of expression.
Maybe they might want to send a code using the frames, BRB (Blue Red Blue = Be Right Back)
Another expression like - holding to the frame for longer : Making the LED flash for 5 second could relay that "5 seconds not being with you feel like 5 hours".
We also thought about the moment in time when the recipient is away and not getting informed about the interaction at all. Right now we are debating how to relay that information, or may be not relay at all.

Start the Conversation

Share this Project

Found In

Diot 2019: Social Objects - Part 3

Courses

49713 Designing for the Internet of Things

· 18 members
A hands-on introductory course exploring the Internet of Things and connected product experiences.

Focused on

IoT Products

About

This project helps people "stay in touch" with their loved ones who are living at a distance from each other. By subtly augmenting an everyday object; the picture frame, within its natural surroundings, we aim to create opportunities to magically convey the message of being missed from one end to the other.

Created

February 19th, 2019

Made by Seema Bansal, Satyan Chawla and Cem Ergin