Outcome

Move up Delete Move down

Intention

The main reason I sought to build this project was to promote circular and sharing economy practices. As a person who has moved through 5 different cities across 2 countries in the past year, I can say from personal experience that it is a very painful process to setup a place, use it and finally, find people to pass on the things that one initially started with. For instance - mattresses, table, chair, bed frame, lamps, even clothes that are in a good condition. Another thought behind this was the model of sharing economy, suppose I want to use a drilling machine to drill 4/5 holes in my apartment. Now the upfront cost of purchasing this would be too much, but I wouldn't mind paying a few dollars to rent the machine for a few hours. Building up on these experiences, I sought to build a chatbot which can enable this.

Move up Delete Move down

Process

The following points given below explain the process of developing the bot. Although many companies have their own shopping bots, this bot connects buyers and sellers, thereby promoting the sharing economy. Anyone can sell any product at anytime by uploading a picture of it.

Move up Delete Move down

1. Bot Conversation Flow

This is the 'ideal' way a conversation is supposed to happen through the Accio bot. There would be many border cases, and also some situation in which sequential questions need to be asked to users, along with sufficient help along the way.

Move up Delete Move down

Flowchart accio2

Show Advanced Options

Move up Delete Move down

2. Bot Personality

Accio is aimed to be a fun, friendly and helpful bot. The main reason to build on the smalltalk feature (offered by Dialogflow) is to handle all cases where users perform actions not intended or perceived as usual in context of the bot for eg. I am bored, I am sleepy, Do not talk to me, Tell me a joke etc.

Move up Delete Move down

Screen shot 2018 10 19 at 6.40.50 pm

Show Advanced Options

Move up Delete Move down

Screen shot 2018 10 19 at 6.41.24 pm

Show Advanced Options

Move up Delete Move down

3. Data Diagram

The following data diagram illustrates how the user data is going to be stored on the server.

Accio_userdetails: records the unique details of a person. Later this same table will be used to reference other tables based on the user_id
Accio_purchases: records the purchase details such as the amount paid and the product that has been purchased.
Accio_products: lists the product price, the quantity purchased and it's price.

Move up Delete Move down

Data diagram

Show Advanced Options

Move up Delete Move down

4. The Ideal Flow for the user interaction with the bot

Previously the bot was called 'buy sell' while testing the interaction in Botsoceity.

Botsoceity is a useful too for mocking up sample interactions with a bot. https://botsociety.io/

Move up Delete Move down

Mebot mockup 8f764dbc9098

Show Advanced Options

Move up Delete Move down

Final Product

Detail what you created. What methods or techniques did you use? What tools and technologies were involved? Include images, code or video.

Move up Delete Move down

1. Gems Used

the following is the code used in the Gemfile

Move up Delete Move down

ruby '2.4.4'

source 'https://rubygems.org'

gem 'sinatra'
gem 'sinatra-contrib'

gem 'dotenv-rails', :groups => [:development, :test]

gem 'twilio-ruby', ">5.0.0"

gem "google-api-client", '~> 0.8'
gem 'google-cloud-dialogflow', '~> 0.2.2'
gem 'rakuten_web_service'

gem 'activerecord'
gem 'pg'
gem 'sinatra-activerecord' # excellent gem that ports ActiveRecord for Sinatra
gem 'rake', '>= 0.9.2'

gem 'json'
gem 'shotgun'

Show Advanced Options

Move up Delete Move down

2. Response Determination and output using app.rb file:

The following code piece is about how the response is given based on the detected intents. By no means it is able to cover use cases, and that being said, the bot is still under development.

Move up Delete Move down

def determine_response (body)
    body = body.downcase.strip
    no = "i didnt quite understand what you mean"
    
    project_id = ENV["GOOGLE_CLOUD_PROJECT"]
    query_result = detect_intent_texts project_id: project_id,
                            session_id: SecureRandom.uuid,
                            texts: [body],
                            language_code:"en-US"
    intent = query_result.intent
    parameters = JSON.parse(query_result.parameters.to_json)
    fullfillment_text = query_result.fulfillment_text
        
    if intent.blank?
      response = fullfillment_text
    else
      intent = intent.display_name
      
      if intent == "EventIntent"
        service = Google::Apis::CalendarV3::CalendarService.new
        service.client_options.application_name = APPLICATION_NAME
        service.authorization = authorize

        # Fetch the next 10 events for the user
        calendar_id = 'primary'
        reply = service.list_events(calendar_id,
                                       max_results: 10,
                                       single_events: true,
                                       order_by: 'startTime',
                                       time_min: Time.now.iso8601)
        puts = 'Upcoming events:'
        puts = 'No upcoming events found' if reply.items.empty?
        reply.items.each do |event|
          start = event.start.date || event.start.date_time
          reply =  "- #{event.summary} (#{start})"
          response = reply
        end
      
      elsif intent == "WelcomeIntent"
        time = Time.new 
        time_today = time.strftime("%A %B %d, %Y %H:%M")
        evening_greetings = ["Toodles", "Namaste", "Accio says hi", "Hej hej", "Hola!", "good evening friend!"]
        morning_greeting = ["good morning", "hello sunshine", "wakey wakey", "good morgon", "lumos"]
        media = "https://media.giphy.com/media/13ZHjidRzoi7n2/giphy.gif"
      
        if time.hour >= 12
            message = evening_greetings.sample 
        elsif time.hour <12
            message = morning_greeting.sample
        end
        response = message
    
      elsif intent == "AboutIntent"
        response = "Hello, I am Accio! I can help you to buy or sell things. If you're stuck, type help."
    
      elsif intent == "SearchIntent"
        items = RakutenWebService::Ichiba::Product.search(keyword: body) # This returns Enumerable object
        items.first(10).each do |item|
          puts "#{item['productName']}, #{item['maxPrice'] * 0.0089} USD" # You can refer to values as well as Hash.
          response = "#{item['productName']}, #{item['maxPrice'] * 0.0089} USD" 
        end
    
      elsif intent == "JokeIntent"
        array_of_lines = IO.readlines("jokes.txt")
        response = array_of_lines.sample
    
      elsif intent == "HelpIntent"
        response = "To buy or sell a product, type buy or sell followed by the name of the product. You can also type Accio followed by the product's name for buying"
      
      elsif intent == "DefaultFallbackIntent"
        response = fullfillment_text
        
      elsif intent == "productNameIntent"
        # puts parameters
        # if parameters.blank?
          # response = fullfillment_text
        # else
        # puts parameters['product_name']
        product_name = parameters['fields']['productName']['listValue']['values']
        number = parameters['fields']['number']['listValue']['values']
        
        puts product_name
        puts number 
        if !product_name.empty?
          product_name = product_name[0]['stringValue'].to_s
        end
        if !number.empty?
          number = number[0]['numberValue'].to_s
        end
        if product_name.empty? || number.empty?
          response = fullfillment_text
        else
          # response = product_name + " " + number
          response = "I found these items : "
          items = RakutenWebService::Ichiba::Product.search(keyword: product_name) # This returns Enumerable object
          items.first(5).each do |item|
            puts item.to_s
            puts "#{item['productName']}, #{item['maxPrice'] * 0.0089 * number.to_i} USD" # You can refer to values as well as Hash.
            response = response + "
            #{item['productName']}, #{item['maxPrice'] * 0.0089 * number.to_i} USD
            #{item['productUrlMobile']}
            " 
          end
        end
        # end
      else
        response = intent + " " + fullfillment_text
      end 
    end
    return response
end

Show Advanced Options

Move up Delete Move down

3. Database: Listing tasks in the REST/ CRUD approach

Move up Delete Move down

get '/accio_userdetails'  do
  AccioUserdetail.all.to_json
end

get '/accio_userdetails/create'  do
  user = {}
  user['name'] = "Harshika Jain"
  user['location'] = "Pittsburgh"
  user['phone_number'] = "+7163308752"
  userOut = AccioUserdetail.new(user)

   if userOut.save
     userOut.to_json
   else
     halt 422, userOut.errors.full_messages.to_json
   end
end

get '/accio_purchases'  do
  AccioPurchase.all.to_json
end

get '/accio_purchases/create'  do
  purchase = {}
  purchase['amount_paid'] = 500
  purchase['paid_by_id'] = 4
  purchase['product_id'] = 71
  purchaseOut = AccioPurchase.new(purchase)

   if purchaseOut.save
     purchaseOut.to_json
   else
     halt 422, purchaseOut.errors.full_messages.to_json
   end
end

Show Advanced Options

Move up Delete Move down

Accio Bot Video

Please watch the video in fullscreen mode

Move up Delete Move down

Show Advanced Options

Move up Delete Move down

Reflection

I personally think that this project was too large to be built in a mini and I might require another mini or say 2 weeks to get a full blown working prototype.

Move up Delete Move down

1. Milestones Ahead:

To detect intents more smoothly, currently, have to dig into Dialogflow interaction logs files to check which parameters are missing and which ones are populating data to the user via Twilio
Dialogflow offers sentiment analysis, how can that be used to help users when they tell accio that they’re bored or they’re happy. (apparently dialogflow agent is responding to those queries but am unable to populate it on the screen)
Replacing rakuten API with craigslist scraper or amazon product search. Later replacing the API with an actual product seller.
Showing product images captured from url in the form of cards (use messenger as a platform maybe..)
Migrate to a platform better suited for online product exchange such as Google Assistant.

Move up Delete Move down

2. Learning from Dialogflow Integration:

When I first experimented with Dialogflow, I ended up hooking it just to Twilio due to which intent detection and answers were being given by Dialogflow itself without the user inputs touching the ruby code at all. In the second iteration, Dialogflow was used to pick up intents while the user queries were answered using the Ruby programme.

the dialogflow ruby integration for intent detection can be found here: https://github.com/Harshikerfuffle/flying-unicorns/blob/master/technical_guide.md

Move up Delete Move down

3. Interact with Accio

Please WhatsApp "join eggplant-emu" on number +1 (415) 523 8886

You can upload files of up to 20MB using this form.

Add Rich Text Add Code Block Add Embeddable Content