III Gallery

Outcome

Overview

For newborn babies, one of the main methods to monitor their health is checking that they are regularly feeding and requiring diaper change. In relation to that, mothers who are breast feeding requires a similarly regular schedule to keep their milk supply constant. This requires the mother to also log their pumping schedule and any bottles that the baby receives as a result. And in the haziness of taking care of a newborn, memories become unreliable and apps require a hand to operate the phone, which is less than ideal when holding the baby.

Using Ruby based app that is accessed through Amazon's Echo Dot allow a hands free operation that allows the parents to log the information as they take care of the baby.

Move up Delete Move down

Feature List

Log every breast-feeding– Time and log every feeding.
Log feeding details – Log which breast, overall experience, and latch quality.
Log bottle feeding detail – Log time and amount of how much formula the baby ate.
Log pumping sessions – Time and log breast pumping detail, including which breast and how much was pumped.
Log diaper change – Time and log each diaper change, type, and quality.
Provide information – give previous feeding, pumping, or diaper change session upon request.

Move up Delete Move down

Functionality Matrix

Move up Delete Move down

Show Advanced Options

Move up Delete Move down

Workflow Diagram with Data Diagram

Move up Delete Move down

Show Advanced Options

Move up Delete Move down

Ruby Code

Move up Delete Move down

require 'json'
require "sinatra"
require 'active_support/all'
require "active_support/core_ext"
require 'sinatra/activerecord'
require 'rake'
require 'active_support/core_ext/time'
require 'time'

require 'twilio-ruby'

require 'alexa_skills_ruby'
require 'httparty'
require 'haml'
require 'iso8601'

# set :database, "sqlite3:db/bab-e.db"

require_relative './models/bottle'
require_relative './models/breast'
require_relative './models/diaper'
require_relative './models/pumping'
require_relative './models/user'

# Load environment variables using Dotenv. If a .env file exists, it will
# set environment variables from that file (useful for dev environments)
configure :development do
  require 'dotenv'
  Dotenv.load
end

# enable sessions for this project

enable :sessions

# First you'll need to visit Twillio and create an account 
# you'll need to know 
# 1) your phone number 
# 2) your Account SID (on the console home page)
# 3) your Account Auth Token (on the console home page)
# then add these to the .env file 
# and use 
#   heroku config:set TWILIO_ACCOUNT_SID=XXXXX 
# for each environment variable

#client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]


# Hook this up to your Webhook for SMS/MMS through the console

get '/' do
  @status = StatusUpdate.order( created_at: 'DESC' ).first
  haml :index
end

post '/' do
  content_type :json

  handler = CustomHandler.new(application_id: ENV['APPLICATION_ID'], logger: logger)

  begin
    handler.handle(request.body.read)
  rescue AlexaSkillsRuby::InvalidApplicationId => e
    logger.error e.to_s
    403
  end

end

class CustomHandler < AlexaSkillsRuby::Handler

    on_intent("Setup") do
        response.set_output_speech_text("Welcome, what is your first name?")
    end
            
    on_intent("SetupFirstName") do
            user = User.new
            user.fname = request.intent.slots["fname"]
            user.save
            response.set_output_speech_text("Your first name is #{user.fname}. What is your last name?")
            #Add confirmation
    end
    
    on_intent("SetupLastName") do
            user = User.last
            user.lname = request.intent.slots["lname"]
            user.save
            response.set_output_speech_text("Your last name is #{user.lname}. What is your baby's name?")
    end
    
    on_intent("SetupBabyName") do
            user = User.last
            user.bname = request.intent.slots["bname"]
            user.save
            response.set_output_speech_text("Your baby's name is #{user.bname}. What is the baby's gender?")
    end

    on_intent("SetupBabyGender") do
            user = User.last
            gender = request.intent.slots["gender"]
            if gender == "girl"
                user.gender = 1
                user.save
                response.set_output_speech_text("Your baby is a girl. Please set your password.")
                
                elsif gender == "boy"
                user.gender = 2
                user.save
                response.set_output_speech_text("Your baby is a boy. Please set your password.")
            end
    end
       
    on_intent("SetupPassword") do
            user = User.last
            user.password = request.intent.slots["password"]
            user.save
            if user.gender == 1
                gender = "girl"
                pronoun = "she"
                
                elsif user.gender == 2
                gender = "boy"
                pronoun = "he"
            end
            #add check
            response.set_output_speech_text("Great, you are all set. Your name is #{user.fname} #{user.lname}. Your baby's name is #{user.bname} and #{pronoun} is a #{gender}.")
    end

#=====================================BREAST FEEDING MODULE =====================================  
    
    on_intent("BeginBreastFeeding") do
        response.set_output_speech_text("Okay, which side is she breast feeding on?")
    end
    
    on_intent("SetBreastSide") do
        user = User.last
            if user.gender == 1
                gender = "girl"
                pronoun = "she"
                
                elsif user.gender == 2
                gender = "boy"
                pronoun = "he"
            end
        side = request.intent.slots["side"]
        # add some validation to check the side is left or right later
        #create the object
        breast = Breast.new
        #add the current time to the start time column
        # this should be a datetime type
        breast.side = side
        t = Time.now
        breast.start = t + Time.zone_offset('EST')
        # save it and update the database with the change
        breast.save

        response.set_output_speech_text("Great, I started the timer for the #{side} side. Say 'breast feeding done' when #{pronoun} stops feeding")
    end
    
    on_intent("EndBreastFeeding") do
        user = User.last
            if user.gender == 1
                gender = "girl"
                pronoun = "she"
                
                elsif user.gender == 2
                gender = "boy"
                pronoun = "he"
            end
        #create the object
        # search for the records in the database that match the side
        # and haven't got a stop time
        breast = Breast.where( end: nil ).first
        # you might also want to add a little more to check a time range
        # e.g. you don't want to update it if its from yesterday, etc.
        # check we have something in the database 
        # i.e. we've got an object to work with 
        unless breast.nil? 
            t = Time.now
            breast.end = t + Time.zone_offset('EST')
              # save it and update the database with the change
            breast.save
            minutes = (breast.end - breast.start)/60
            duration = minutes.round
        end 
        response.set_output_speech_text("I updated that. #{user.bname} fed for #{duration} minutes. How would you rate the quality of the experience from 1 being bad to 10 being great?")
    end
    
    on_intent("BreastFeedingQuality") do
        user = User.last
            if user.gender == 1
                gender = "girl"
                pronoun = "she"
                
                elsif user.gender == 2
                gender = "boy"
                pronoun = "he"
            end
        
        quality = request.intent.slots["rating"]
        
        breast = Breast.where( quality: nil).first
        
        unless breast.nil?
            breast.quality = quality
            breast.save
            side = breast.side
            minutes = (breast.end - breast.start)/60
            duration = minutes.round
            response.set_output_speech_text("Great, I logged that #{pronoun} fed for #{duration} minutes on the #{side} and the experience was rated #{breast.quality}")
        end
    end

    
#================================== BOTTLE FEEDING MODULE =====================================    
    on_intent("BeginBottleFeeding") do
        user = User.last
            if user.gender == 1
                gender = "girl"
                pronoun = "she"
                
                elsif user.gender == 2
                gender = "boy"
                pronoun = "he"
            end
        
        bottle = Bottle.new
        t = Time.now
        bottle.start = t + Time.zone_offset('EST')
        bottle.save
        
        response.set_output_speech_text("Okay, I will begin the timer. How much is #{pronoun} being fed?")
    end
    
    on_intent("BottleAmount") do
        user = User.last
            if user.gender == 1
                gender = "girl"
                pronoun = "she"
                
                elsif user.gender == 2
                gender = "boy"
                pronoun = "he"
            end
        
        bottle = Bottle.last
        
        unless bottle.nil?
            bottle.amount = request.intent.slots["amount"]
            bottle.save!
        end
        
        response.set_output_speech_text("Great, she's feeding #{bottle.amount}oz of milk. Say 'bottle feeding done' when #{pronoun} stops feeding.")
    end
    
    on_intent("EndBottleFeeding") do
        bottle = Bottle.last
        t = Time.now
        bottle.end = t + Time.zone_offset('EST')
        bottle.save
        
            user = User.last
            if user.gender == 1
                gender = "girl"
                pronoun = "she"
                
                elsif user.gender == 2
                gender = "boy"
                pronoun = "he"
            end
        
        unless bottle.nil?
            minutes = (bottle.end - bottle.start)/60
            duration = minutes.round
        
            quantity = bottle.amount
        
            response.set_output_speech_text("Ok, #{pronoun} fed for #{duration} minutes. #{user.bname} drank #{quantity}ounce of milk.")
        end
    end
    
#================================== FEEDING QUERY =====================================    
    
    on_intent("LastFeeding") do
        breast = Breast.last
        bottle = Bottle.last
        
        if breast.end > bottle.end
            time = breast.start.strftime( "%A %e at %l %M %P" )
            side = breast.side
            minutes = (breast.end - breast.start)/60
            duration = minutes.round
            quality = breast.quality
            
            user = User.last
            if user.gender == 1
                gender = "girl"
                pronoun = "she"
                
                elsif user.gender == 2
                gender = "boy"
                pronoun = "he"
            end
        
        response.set_output_speech_text("#{user.bname} was breast fed on #{side} side at #{time} for #{duration} minutes. The experience was rated #{quality}")  
        
        elsif bottle.end > breast.end
            time = bottle.start.strftime( "%A %e at %l %M %P" )
            amount = bottle.amount
            minutes = (bottle.end - bottle.start)/60
            duration = minutes.round
            
            user = User.last
            if user.gender == 1
                gender = "girl"
                pronoun = "she"
                
                elsif user.gender == 2
                gender = "boy"
                pronoun = "he"
            end
            
        response.set_output_speech_text("#{user.bname} was bottle fed with #{amount}oz at #{time} for #{duration} minutes." )        
        end
    end

#===================================== PUMPING MODULE =====================================    
    on_intent("BeginPumping") do
        pumping = Pumping.new
        t = Time.now
        pumping.start = t + Time.zone_offset('EST')
        pumping.save
        response.set_output_speech_text("Great, what side are you starting the pump on?")
    end
    
    on_intent("PumpingSide") do
            pumping = Pumping.last
            pumping.side = request.intent.slots["side"]
            pumping.save
            response.set_output_speech_text("ok, I've recorderd that you are pumping on the #{pumping.side}. Let me know when you finish by saying 'pumping done'.")        
    end
    
    on_intent("EndPumping") do
            pumping = Pumping.last
            t = Time.now
            pumping.end = t + Time.zone_offset('EST')
            pumping.save
                unless pumping.nil?
                    minutes = (pumping.end - pumping.start)/60
                    duration = minutes.round
                    response.set_output_speech_text("Ok, you pumped on the #{pumping.side} for #{duration}. How much did you end up pumping?")             
                end
    end
    
    on_intent("PumpingAmount") do
        pumping = Pumping.last
            pumping.amount = request.intent.slots["amount"]
            pumping.save
                unless pumping.nil?
                    minutes = (pumping.end - pumping.start)/60
                    duration = minutes.round
                    response.set_output_speech_text("Great job, you pumped for #{duration} minutes on the #{pumping.side} side. You pumped #{pumping.amount}ounces.")
                end
    end
    
    on_intent("LastPumping") do
        pumping = Pumping.last
            unless pumping.nil?
                    minutes = (pumping.end - pumping.start)/60
                    duration = minutes.round
                    response.set_output_speech_text("You pumped for #{duration} minutes on the #{pumping.side} side. You pumped #{pumping.amount}oz.")
            end
    end

#===================================== DIAPER LIST =====================================    
    on_intent("DChange") do
        user = User.last
            if user.gender == 1
                gender = "girl"
                pronoun = "she"
                
                elsif user.gender == 2
                gender = "boy"
                pronoun = "he"
            end
        
        diaper = Diaper.new
        t = Time.now
        diaper.start = t + Time.zone_offset('EST')
        diaper.save
        response.set_output_speech_text("Ok, what did #{user.bname} have in the diaper?")
    end
    
    on_intent("DType") do
        diaper = Diaper.last
        time = diaper.start.strftime ( "%A %e at %l %M %P" )
        diaper.save
        body = request.intent.slots["type"]
        
            if body == "pee"
                user = User.last
                    if user.gender == 1
                        gender = "girl"
                        pronoun = "she"

                        elsif user.gender == 2
                        gender = "boy"
                        pronoun = "he"
                    end

                diaper.dtype = "1"
                diaper.save
                response.set_output_speech_text("Great, I logged that #{pronoun} had pee diaper at #{time}.")

                elsif body == "poo"
                    user = User.last
                        if user.gender == 1
                            gender = "girl"
                            pronoun = "she"

                            elsif user.gender == 2
                            gender = "boy"
                            pronoun = "he"
                        end
                
                diaper.dtype = "2"
                diaper.save
                response.set_output_speech_text("Great, I logged that #{pronoun} had poo diaper at #{time}.")

                elsif body == "both"
                    user = User.last
                    if user.gender == 1
                        gender = "girl"
                        pronoun = "she"

                        elsif user.gender == 2
                        gender = "boy"
                        pronoun = "he"
                    end
                    
                diaper.dtype = "3"
                diaper.save
                response.set_output_speech_text("Great, I logged that #{pronoun} had diaper with both types at #{time}.")
            end
    end
    
    on_intent("LastDiaper") do
       diaper = Diaper.last
        user = User.last
        time = diaper.start.strftime ( "%A %e at %l %M %P" )
        
        if diaper.dtype == 1
            response.set_output_speech_text("#{user.bname} had a pee diaper at #{time}.")
            
            elsif diaper.dtype == 2
            response.set_output_speech_text("#{user.bname} had a poo diaper at #{time}.")
            
            elsif diaper.dtype == 3
            response.set_output_speech_text("#{user.bname} had a with both at #{time}.")
        end
    end
    
end

Show Advanced Options

Move up Delete Move down

Utterance

Move up Delete Move down

Setup setup

Setup user setup

SetupFirstName my name is {fname}

SetupFirstName my first name is {fname}

SetupLastName my last name is {lname}

SetupLastName my family name is {lname}

SetupBabyName my baby's name is {bname}

SetupBabyName her name is {bname}

SetupBabyName his name is {bname}

SetupBabyGender my baby is a {gender}

SetupBabyGender she is a {gender}

SetupBabyGender he is a {gender}

SetupPassword my password is {password}

SetupPassword my password will be {password}

SetupPassword make my password {password}

BeginBreastFeeding started breast feeding

BeginBreastFeeding is breast feeding

BeginBreastFeeding I started breast feeding

BeginBreastFeeding she is breast feeding

BeginBreastFeeding he is breast feeding

SetBreastSide feeding on the {side} side

SetBreastSide is feeding on the {side} side

SetBreastSide feeding on {side} side

SetBreastSide she is on the {side} side

SetBreastSide he is on the {side} side

EndBreastFeeding breast feeding done

EndBreastFeeding done breast feeding

EndBreastFeeding breast done

EndBreastFeeding she finished breast feeding

EndBreastFeeding he finished breast feeding

BreastFeedingQuality {rating}

BreastFeedingQuality the experience was {rating}

BreastFeedingQuality I would rate it {rating}

BeginBottleFeeding start bottle feeding

BeginBottleFeeding feeding on the bottle

BeginBottleFeeding begin bottle feeding timer

BeginBottleFeeding is drinking a bottle

BeginBottleFeeding is drinking a bottle

BeginBottleFeeding is being bottle fed

BeginBottleFeeding is being bottle fed

BottleAmount is drinking {amount} ounce.

BottleAmount is drinking {amount} ounce.

BottleAmount is feeding {amount} ounce.

BottleAmount had {amount} ounce.

BottleAmount had {amount} ounce.

EndBottleFeeding bottle done

EndBottleFeeding bottle feeding done

EndBottleFeeding she finished the bottle

EndBottleFeeding he finished the bottle

EndBottleFeeding she is done with the bottle

EndBottleFeeding he is done with the bottle

LastFeeding last feeding

LastFeeding check last feeding

LastFeeding when was the last feeding

LastFeeding when did she feed last

LastFeeding when did he feed last

BeginPumping begin pumping

BeginPumping I begin pumping

BeginPumping begin pumping timer

PumpingSide I am pumping on the {side} side

PumpingSide pumping on the {side} side

EndPumping pumping done

EndPumping pumping finished

EndPumping end pumping timer

EndPumping I'm done pumping

PumpingAmount I pumped {amount} ounce

PumpingAmount I got {amount} ounce

LastPumping last pumping

LastPumping when was the last pumping

LastPumping when did I last pump

DChange diaper change

DChange diaper

DChange has a diaper change

DType she had a {type} in the diaper

DType he had a {type} in the diaper

DType there was a {type}

DType it was a {type}

DType has a {type} diaper

LastDiaper last diaper

LastDiaper when was the last diaper

LastDiaper what was the last diaper

Show Advanced Options

Move up Delete Move down

Intent Schema

Move up Delete Move down

{
  "intents": [
    {
      "intent":"Setup"
    },
    {
      "intent": "SetupFirstName",
        "slots":[
            {
                "name": "fname",
                "type": "NAME_LIST"
            }
            ]
    },
    {
      "intent": "SetupLastName",
      "slots":[
            {
                "name": "lname",
                "type": "NAME_LIST"
            }
        ]
    },
    {
      "intent": "SetupBabyName",
      "slots":[
            {
                "name": "bname",
                "type": "NAME_LIST"
            }
        ]
    },
    {
      "intent": "SetupBabyGender",
      "slots":[
            {
                "name": "gender",
                "type": "GENDER_LIST"
            }
        ]
    },
    {
      "intent": "SetupPassword",
      "slots":[
            {
                "name": "password",
                "type": "PASSWORD_LIST"
            }
        ]
    },
    {
      "intent": "BeginBreastFeeding"
    },
    {
      "intent": "SetBreastSide",
      "slots":[
            {
                "name": "side",
                "type": "SIDE_LIST"
            }
        ]
    },
    {
      "intent": "EndBreastFeeding"
    },
    {
      "intent": "BreastFeedingQuality",
      "slots":[
            {
                "name": "rating",
                "type": "AMAZON.NUMBER"
            }
        ]
    },
    {
      "intent": "BeginBottleFeeding"
    },
    {
      "intent": "BottleAmount",
      "slots":[
            {
                "name": "amount",
                "type": "AMAZON.NUMBER"
            }
        ]
    },
    {
      "intent": "EndBottleFeeding"
    },
    {
      "intent": "LastFeeding"
    },
    {
      "intent": "BeginPumping"
    },
    {
      "intent": "PumpingSide",
      "slots":[
            {
                "name": "side",
                "type": "SIDE_LIST"
            }
        ]
    },
    {
      "intent": "EndPumping"
    },
    {
      "intent": "PumpingAmount",
      "slots":[
            {
                "name": "amount",
                "type": "AMAZON.NUMBER"
            }
        ]
    },
    {
      "intent": "DiaperType",
      "slots":[
            {
                "name": "type",
                "type": "TYPE_LIST"
            }
        ]
    }
  ]
}

Show Advanced Options

Move up Delete Move down

Show Advanced Options

Move up Delete Move down

Future Work/Considerations

A large number of words are shared between different functions which throws Alexa off, leading to different functions being called. There are two methods to resolve this. One is to continue adding to the utterance, being mindful about the length and shared slots. The other is to run a couple of voice training on Alexa itself to increase it's accuracy.

In the future, there are a number of functionalities that can be added. These include printing the current db to json on html page, creating a reminder function, password lock/protection, and ability to delete records in case of mistakes.

You can upload files of up to 20MB using this form.

Add Rich Text Add Code Block Add Embeddable Content

Outcome

Overview

Feature List

Functionality Matrix

Attribution

Workflow Diagram with Data Diagram

Attribution

Ruby Code

Attribution

Utterance

Attribution

Intent Schema

Attribution

Attribution

Future Work/Considerations