Outcome

Move up Delete Move down

We had a baby daughter this fall. With my busy schedule of work and class, I felt not very connected to my daughters activities throughout the day. This is when I saw an opportunity to pursue this project using alexa and an sms bot to help me track my daughters activities throughout the day and thereby feel connected.

Move up Delete Move down

This project was done in four steps:

1. Conceptualization

2. Technical exploration

3. Experience prototype

4. Implementation

Move up Delete Move down

Step 1: Conceptualization

Here I learnt to ideate and create a personality for my Bot and also the general workflow/structure of the conversation.

Move up Delete Move down

Outcome:

1. Bot personality design

2. Workflow of the conversation with the alexa skill and sms bot

3. Priority matrix

Move up Delete Move down

Bot_personality.pdf

549.450 KB · Download / View

Show Advanced Options

Move up Delete Move down

Pop aabio workflow

Show Advanced Options

Move up Delete Move down

Pop priority matrix

Show Advanced Options

Move up Delete Move down

Step 2: Technical exploration

In this step I took the conceptual ideas and did a technical exploration to see how feasible it is get this idea implemented. This includes doing research to see if there are other products out there that are similar, searching for API's that could help and etc.

Move up Delete Move down

Outcome:

1. Data model (Entity relationship diagram)

2. Technical Precedents - list the relevant examples and gems

Move up Delete Move down

Img 7953 %281%29

Show Advanced Options

Move up Delete Move down

Technical Precedents

Alexa baby stats skill (Links to an external site.)Links to an external site. - This is very close to what I wanted to create. The only missing component is an sms companion app. I also feel this skill is bloated, and would like to trim down a lot its functionality

Infant weight data (Links to an external site.)Links to an external site.API- This would allow us to plotting growth charts

Global data chart on child growth by WHO (Links to an external site.)Links to an external site.

How to query UN dataset using REST API (Links to an external site.)Links to an external site. - This link teaches how to use to query information from UN dataset

How to design a chatbot personality (Links to an external site.)Links to an external site. - This chatbot magazine article provides guidelines to design a personality for chatbots

HOw NLP helps creating a personality (Links to an external site.)Links to an external site. - Article that describes how NLP helps process the user intent and use it to respond with a personality

Baby personality types (Links to an external site.)Links to an external site. - This article describes various baby personality types and for our SMS bot we can choose personality type based on type of message.

Move up Delete Move down

Step 3: Experience prototype

In this step create a quick prototype in code. Instead of building all the bells and whistles, create a first cut version with fake date to make it look functional. This is used to just test out the concept with other students in the class and get feedback on the experience. Here is the code of that experience prototype.

Move up Delete Move down

require "sinatra"
require 'sinatra/reloader' if development?

require 'alexa_skills_ruby'
require 'httparty'
require 'iso8601'

require 'twilio-ruby'
require 'giphy'

# ----------------------------------------------------------------------

# Load environment variables using Dotenv. If a .env file exists, it will
# set environment variables from that file (useful for dev environments)
configure :development do
  require 'dotenv'
  Dotenv.load
end

# enable sessions for this project
enable :sessions

@client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]




# ----------------------------------------------------------------------
#     How you handle your Alexa
# ----------------------------------------------------------------------

class CustomHandler < AlexaSkillsRuby::Handler

  on_intent("RecordFeedIntent") do
    response.set_output_speech_text("Well fed baby, happy baby")

    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_gif_for ("feeding baby")

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: " 👶 🍼 Just had a nice feed 👼 ",
      media_url: gif_url
    )

    logger.info 'RecordFeedIntent processed'
  end

  on_intent("RecordDiaperChangeIntent") do
    response.set_output_speech_text("your baby is feeling much better")

    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_gif_for ("diaper change")

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: " 👶 Got my diapers changed, feeling better ✌️",
      media_url: gif_url
    )

    logger.info 'RecordDiaperChangeIntent processed'

  end

  on_intent("RecordSleepIntent") do
    response.set_output_speech_text("sleep well baby")

    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_gif_for ("sleeping baby")

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: "👼 💤 I am sleeping 🔇*DO NOT DISTURB*📵 ",
      media_url: gif_url
    )

    logger.info 'RecordFeedIntent processed'
  end

  on_intent("AMAZON.HelpIntent") do

    response.set_output_speech_text("Howdy! I am Aabio, a skill that will record your baby stats, like feed, sleep and diaper change")
    logger.info 'AMAZON.HelpIntent processed'

  end



end

# ----------------------------------------------------------------------
#     ROUTES, END POINTS AND ACTIONS
# ----------------------------------------------------------------------


get '/' do
  "Hello from Aabio"
end



post '/alexa/incoming' do
  content_type :json

  handler = CustomHandler.new(application_id: ENV['ALEXA_APPLICATION_ID'], logger: logger)

  begin
    hdrs = { 'Signature' => request.env['HTTP_SIGNATURE'], 'SignatureCertChainUrl' => request.env['HTTP_SIGNATURECERTCHAINURL'] }
    handler.handle(request.body.read, hdrs)
  rescue AlexaSkillsRuby::Error => e
    logger.error e.to_s
    403
  end

end


get "/sms/incoming" do
  session["last_intent"] ||= nil

  session["counter"] ||= 1
  count = session["counter"]

  sender = params[:From] || ""
  body = params[:Body] || ""
  body = body.downcase.strip

  message =""
  media = nil

  if is_help_intent? body
    message = get_help_message
  elsif is_casual_intent? body
      message = get_greeting
  elsif body == "status" or body.include? "status"
    message = get_status
  elsif body == "how are you" or body.include? "feel"
    FEEL = ["Happy baby", "Sad baby", "Baby smile", "Baby cry"]
    message = FEEL.sample
    media = get_gif_for(message)
  elsif body == "how many poops today" or body.include? "poop" or body.include? "💩"
    POOP = ["1💩", "2💩💩", "3💩💩💩", "4💩💩💩💩", "5💩💩💩💩💩","6💩💩💩💩💩💩"]
    message = POOP.sample
  elsif body == "how many feed today"  or body.include? "feed" or body.include? "🍼"
    FEED = ["1🍼", "2🍼🍼", "3🍼🍼🍼", "4🍼🍼🍼🍼", "5🍼🍼🍼🍼🍼", "6🍼🍼🍼🍼🍼🍼"]
    message = FEED.sample
  elsif body == "how much sleep today" or body.include? "sleep" or body.include? "⌛"
    SLEEP = ["😴1hr⌛", "😴2hr⌛⌛", "😴3hr⌛⌛⌛", "😴4hr⌛⌛⌛⌛"]
    message = SLEEP.sample

  else
    media = get_gif_for ("confused baby")
    message = "I am confused. Type [poop] or [sleep] or [feed] or [Status] or [How are you]"
  end


  twiml = Twilio::TwiML::MessagingResponse.new do |r|
    r.message do |m|
      m.body( message )
      unless media.nil?
        m.media( media )
      end
    end
  end


  content_type 'text/xml'
  twiml.to_s

end

# Casual intent
CASUAL_INTENTS = ["hi", "hello", "yo", "whatsup", "what's up", "hey"]

def is_casual_intent? incoming_text

  CASUAL_INTENTS.each do |word|
    if incoming_text.start_with?(word)
      return true
    end
  end

  return false

end

GREETINGS = ["Hi","Yo", "Hey","Howdy", "Hello", "Ahoy", "‘Ello", "Aloha", "Hola", "Bonjour", "Hallo", "Ciao", "Konnichiwa", "Vanakam!"]
INTROS = ["I am Aabio, a baby bot that is going to replicate your baby. You can communicate with me as you would with your baby and I will do my best to respond. Some sample commands are [Status],[How are you]"]

def get_greeting
  GREETINGS.sample + ", " + INTROS.sample
end


# Help intent
HELP_INTENTS = ["Help", "I am confused", "H", "Please Help", "What", "What are you"]

def is_help_intent? incoming_text

  HELP_INTENTS.each do |word|
    if incoming_text.start_with?(word)
      return true
    end
  end

  return false

end

def get_help_message
  "Howdy! I am Aabio, a bot that is going to replicate your baby. You can communicate with me as you would with your baby and I will do my best to respond."
end

#=========================================================
#Status intent
#=========================================================
POOP = ["1💩", "2💩💩", "3💩💩💩", "4💩💩💩💩", "5💩💩💩💩💩","6💩💩💩💩💩💩"]
FEED = ["1🍼", "2🍼🍼", "3🍼🍼🍼", "4🍼🍼🍼🍼", "5🍼🍼🍼🍼🍼", "6🍼🍼🍼🍼🍼🍼"]
SLEEP = ["😴1hr⌛", "😴2hr⌛⌛", "😴3hr⌛⌛⌛", "😴4hr⌛⌛⌛⌛"]

def get_status
  POOP.sample + ", " + FEED.sample + ", " +SLEEP.sample
end

# THE APPLICATION ID CAN BE FOUND IN THE


get "/test/:action" do

  if params[:action] == "feed"

    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_trending_gif

    puts gif_url

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: "feeding gif",
      media_url: gif_url
    )

    "Recorded feeding and sent a random gif to your phone"

  elsif params[:action] == "diaper"
    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_trending_gif

    puts gif_url

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: "diaper gif",
      media_url: gif_url
    )

    "Recorded diaper change and sent a random gif to your phone"


  elsif params[:action] == "sleep"

    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_trending_gif

    puts gif_url

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: "sleeping gif",
      media_url: gif_url
    )

    "Recorded sleeping and sent a random gif to your phone"


  else
    #slots = request.intent.slots
    "Howdy! I am Aabio, a skill that will record your baby stats, like feed, sleep and diaper change"
  end

end



# ----------------------------------------------------------------------
#     ERRORS
# ----------------------------------------------------------------------
error 401 do
  "Not allowed!!!"
end



#--------------------------------------------------
#--------------------------------------------------
#--------------------------------------------------
#
# =>  GIPHY
#
#--------------------------------------------------
#--------------------------------------------------
#--------------------------------------------------

def get_gif_for query


  Giphy::Configuration.configure do |config|
    config.api_key = ENV["GIPHY_API_KEY"]
  end

  results = Giphy.search( query, {limit: 10})
  gif = nil

  #puts results.to_yaml
  unless results.empty?
    gif = results.sample.fixed_width_downsampled_image.url.to_s
  end

  gif

end



def get_trending_gif


  Giphy::Configuration.configure do |config|
    config.api_key = ENV["GIPHY_API_KEY"]
  end

  results = Giphy.trending(limit: 10)
  gif = nil

  #puts results.to_yaml
  unless results.empty?
    gif = results.sample.fixed_width_downsampled_image.url.to_s
  end

  gif

end

Show Advanced Options

Move up Delete Move down

Step 4: Implementation

This is the final implementation with as much functionality I could add with database and using live date. This code is shown below.

Move up Delete Move down

require "sinatra"
require 'sinatra/reloader' if development?

require 'alexa_skills_ruby'
require 'httparty'
require 'iso8601'

require 'twilio-ruby'
require 'giphy'

require 'sinatra/activerecord'
require 'rake'
require 'json'

require 'active_support/all'
require "active_support/core_ext"

require 'date'


# ----------------------------------------------------------------------

# Load environment variables using Dotenv. If a .env file exists, it will
# set environment variables from that file (useful for dev environments)
configure :development do
  require 'dotenv'
  Dotenv.load
end

# require models
require_relative './models/user'
require_relative './models/sleep_log'
require_relative './models/feed_log'
require_relative './models/poop_log'

# enable sessions for this project
enable :sessions

@client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]




# ----------------------------------------------------------------------
#     How you handle your Alexa
# ----------------------------------------------------------------------

class CustomHandler < AlexaSkillsRuby::Handler

  on_intent("RecordFeedIntent") do
    response.set_output_speech_text("Well fed baby, happy baby")

    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_gif_for ("feeding baby")

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: " 👶 🍼 Just had a nice feed 👼 ",
      media_url: gif_url
    )

    logger.info 'RecordFeedIntent processed'
    #add a record to feed_log
    feed_log.create(phone_number: "+18646337057")
  end

  on_intent("RecordDiaperChangeIntent") do
    response.set_output_speech_text("your baby is feeling much better")

    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_gif_for ("diaper change")

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: " 👶 Got my diapers changed, feeling better ✌️",
      media_url: gif_url
    )

    logger.info 'RecordDiaperChangeIntent processed'

    #add a record to diaper_log
    poop_log.create(phone_number: "+18646337057")
  end

  on_intent("RecordSleepIntent") do
    response.set_output_speech_text("sleep well baby")

    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_gif_for ("sleeping baby")

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: "👼 💤 I am sleeping 🔇*DO NOT DISTURB*📵 ",
      media_url: gif_url
    )

    logger.info 'RecordFeedIntent processed'

    #add a record to sleep_log
    sleep_log.create(phone_number: "+18646337057")
  end

  on_intent("AMAZON.HelpIntent") do

    response.set_output_speech_text("Howdy! I am Aabio, a skill that will record your baby stats, like feed, sleep and diaper change")
    logger.info 'AMAZON.HelpIntent processed'

  end



end

# ----------------------------------------------------------------------
#     ROUTES, END POINTS AND ACTIONS
# ----------------------------------------------------------------------


get '/' do
  "Hello from Aabio"
end



post '/alexa/incoming' do
  content_type :json

  handler = CustomHandler.new(application_id: ENV['ALEXA_APPLICATION_ID'], logger: logger)

  begin
    hdrs = { 'Signature' => request.env['HTTP_SIGNATURE'], 'SignatureCertChainUrl' => request.env['HTTP_SIGNATURECERTCHAINURL'] }
    handler.handle(request.body.read, hdrs)
  rescue AlexaSkillsRuby::Error => e
    logger.error e.to_s
    403
  end

end


get "/sms/incoming" do
  session["last_intent"] ||= nil

  session["counter"] ||= 1
  count = session["counter"]

  sender = params[:From] || ""
  body = params[:Body] || ""
  body = body.downcase.strip

  message =""
  media = nil

  if is_help_intent? body
    message = get_help_message
  elsif is_casual_intent? body
      message = get_greeting
  elsif body == "status" or body.include? "status"
    message = get_status
  elsif body == "how are you" or body.include? "feel"
    FEEL = ["Happy baby", "Sad baby", "Baby smile", "baby cry"]
    # message = FEEL.sample

    how_many_poops = PoopLog.where( phone_number: sender ).where( "created_at > ?" , Time.now - 4.hours ).count
    how_many_bottles = FeedLog.where( phone_number: sender ).where( "created_at > ?" , Time.now - 4.hours ).count

    if how_many_poops > 3
      message = "I'm feeling shitty"
    end
    elsif how_many_bottles > 3
      message = "I'm hungry today"
    end

    media = get_gif_for(message)

  elsif body == "how many poops today" or body.include? "poop" or body.include? "💩"
    POOP = ["1💩", "2💩💩", "3💩💩💩", "4💩💩💩💩", "5💩💩💩💩💩","6💩💩💩💩💩💩"]

    how_many_poops = PoopLog.where( phone_number: sender ).where( "created_at > ?" , Time.now - 24.hours ).count

    message = "#{how_many_poops}" + ("💩"*how_many_poops)

    #message = POOP.sample
  elsif body == "how many feed today"  or body.include? "feed" or body.include? "🍼"
    FEED = ["1🍼", "2🍼🍼", "3🍼🍼🍼", "4🍼🍼🍼🍼", "5🍼🍼🍼🍼🍼", "6🍼🍼🍼🍼🍼🍼"]
    message = FEED.sample
  elsif body == "how much sleep today" or body.include? "sleep" or body.include? "⌛"
    SLEEP = ["😴1hr⌛", "😴2hr⌛⌛", "😴3hr⌛⌛⌛", "😴4hr⌛⌛⌛⌛"]
    message = SLEEP.sample

  else
    media = get_gif_for ("confused baby")
    message = "Type [poop] or [sleep] or [feed] or [Status] or [How are you]"
  end


  twiml = Twilio::TwiML::MessagingResponse.new do |r|
    r.message do |m|
      m.body( message )
      unless media.nil?
        m.media( media )
      end
    end
  end


  content_type 'text/xml'
  twiml.to_s

end

# Casual intent
CASUAL_INTENTS = ["hi", "hello", "yo", "whatsup", "what's up", "hey"]

def is_casual_intent? incoming_text

  CASUAL_INTENTS.each do |word|
    if incoming_text.start_with?(word)
      return true
    end
  end

  return false

end

GREETINGS = ["Hi","Yo", "Hey","Howdy", "Hello", "Ahoy", "‘Ello", "Aloha", "Hola", "Bonjour", "Hallo", "Ciao", "Konnichiwa", "Vanakam!"]
INTROS = ["I am Aabio, a baby bot that is going to replicate your baby. You can communicate with me as you would with your baby and I will do my best to respond. Some sample commands are [Status],[How are you]"]

def get_greeting
  GREETINGS.sample + ", " + INTROS.sample
end


# Help intent
HELP_INTENTS = ["help", "i am confused", "h", "please help", "what", "what are you"]

def is_help_intent? incoming_text

  HELP_INTENTS.each do |word|
    if incoming_text.start_with?(word)
      return true
    end
  end

  return false

end

def get_help_message
  "Howdy! I am Aabio, a bot that is going to replicate your baby. You can communicate with me as you would with your baby and I will do my best to respond."
end

#=========================================================
#Status intent
#=========================================================
POOP = ["1💩", "2💩💩", "3💩💩💩", "4💩💩💩💩", "5💩💩💩💩💩","6💩💩💩💩💩💩"]
FEED = ["1🍼", "2🍼🍼", "3🍼🍼🍼", "4🍼🍼🍼🍼", "5🍼🍼🍼🍼🍼", "6🍼🍼🍼🍼🍼🍼"]
SLEEP = ["😴1hr⌛", "😴2hr⌛⌛", "😴3hr⌛⌛⌛", "😴4hr⌛⌛⌛⌛"]

def get_status
  POOP.sample + ", " + FEED.sample + ", " +SLEEP.sample
end

# THE APPLICATION ID CAN BE FOUND IN THE


get "/test/:action" do

  if params[:action] == "feed"

    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_trending_gif

    puts gif_url

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: "feeding gif",
      media_url: gif_url
    )

    "Recorded feeding and sent a random gif to your phone"

  elsif params[:action] == "diaper"
    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_trending_gif

    puts gif_url

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: "diaper gif",
      media_url: gif_url
    )

    "Recorded diaper change and sent a random gif to your phone"


  elsif params[:action] == "sleep"

    @client = Twilio::REST::Client.new ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"]

    gif_url = get_trending_gif

    puts gif_url

    @client.api.account.messages.create(
      from: ENV['TWILIO_FROM'],
      to: "+18646337057",
      body: "sleeping gif",
      media_url: gif_url
    )

    "Recorded sleeping and sent a random gif to your phone"

  else
    #slots = request.intent.slots
    "Howdy! I am Aabio, a skill that will record your baby stats, like feed, sleep and diaper change"
  end

end



# ----------------------------------------------------------------------
#     ERRORS
# ----------------------------------------------------------------------
error 401 do
  "Not allowed!!!"
end



#--------------------------------------------------
#--------------------------------------------------
#--------------------------------------------------
#
# =>  GIPHY
#
#--------------------------------------------------
#--------------------------------------------------
#--------------------------------------------------

def get_gif_for query


  Giphy::Configuration.configure do |config|
    config.api_key = ENV["GIPHY_API_KEY"]
  end

  results = Giphy.search( query, {limit: 10})
  gif = nil

  #puts results.to_yaml
  unless results.empty?
    gif = results.sample.fixed_width_downsampled_image.url.to_s
  end

  gif

end



def get_trending_gif


  Giphy::Configuration.configure do |config|
    config.api_key = ENV["GIPHY_API_KEY"]
  end

  results = Giphy.trending(limit: 10)
  gif = nil

  #puts results.to_yaml
  unless results.empty?
    gif = results.sample.fixed_width_downsampled_image.url.to_s
  end

  gif

end


# Method to return the current date time

def get_current_time_date
    current_time = DateTime.now
    current_time.strftime "%d/%m/%Y %H:%M"
    # => "14/09/2011 17:02"
  end

Show Advanced Options

Move up Delete Move down

Next Steps:

1. Need to get the database deployed and working

2. Need to add capabilities to send text reminders after a set time of no baby stat being recorded

3. Use gifs of my baby instead of gifpy api.

4. Think of how this could be scaled so, it can be shared as an alexa skill for others to use

You can upload files of up to 20MB using this form.

Add Rich Text Add Code Block Add Embeddable Content