Merge pull request 'Add Entity and Callback Support' (#1) from testing-callback into master

Reviewed-on: #1

README.md

@@ -67,3 +67,56 @@ nlu:
 ```
 
 And, again, you can add more intents here so that Rasa can recognise them. Put these intents in a file called intents.yml and Opsdroid will automatically send that to Rasa to train. 
+
+## Advanced Usage
+
+If you need to have more advanced responses, there are two more features available. You can enable the usage of entities, which allows you to substitute entities detected in the message into the response. You can also define a callback function that can be used to construct a message if more complex behaviour is required.
+
+### Entity Substitution
+
+In order to enable the ability to use entities, you must first add `use_entities: True` under this skill in your Opsdroid configuration file. This will allow the skill to extract the entities and automatically make substitutions. In order for the skill to know where to make the substitutions, we will be using f-string syntax. If an entity is required, but was not extracted by Rasa, then the substitution will fail. At this point, we will use a fallback response. If you wish to customise the possible fallback responses, add these to your Opsdroid config file under `fallback_response_options`.
+
+Note on entity names: In order to have unique keys for each entity, the name of each entity is suffixed with an index representing the order in which the entities occurred. For example, the following input would lead to the following names (assuming Spacy entity extractor):
+
+> What is the weather in Berlin and Dublin in November?
+
+Entity name (key) | Entity value (value)
+--- | ---
+GPE0 | Berlin
+GPE1 | Dublin
+DATE0 | November
+
+Here is an example of a skill config using this feature:
+
+```
+skills:
+        location_repeater:
+                path: "/path/to/skill.py"
+                intent: "location_repeater"
+                use_entities: True
+                response_options:
+                    - "You are in {GPE0}"
+                    - "I think you are in {GPE0}"
+                fallback_response_options:
+                    - "Sorry, I couldn't figure out where you are."
+```
+
+### Callback Creation
+
+If you need more advanced responses, you may want to use the callback feature. Firstly, you need to write a Python (same version that you are using for Opsdroid) function that takes in the message object (see Opsdroid for more documentation on how to use that), and a dictionary of extracted entities. This function should return a string which will be the message to be sent back to the room the request was made in.
+
+Now you will need to add a few lines to your opsdroid configuration file. If you don't enable `use_entities`, then, even if Rasa detects the usage of entities, these will *not* be passed through to your function (None will be sent instead). You then need to provide the name of the python file with your function and the name of the function within that. Here is an example config for the example weather callback provided in this repo:
+
+```
+skills:
+        rasa_weather:
+                path: '/path/to/skill.py'
+                intent: "weather"
+                response_options:
+                        - "Weather request received"
+                use_entities: True
+                fallback_response_options:
+                        - "Sorry, Rasa did not extract sufficient entities to fulfil your request"
+                callback_file: "/path/to/weather_callback.py"
+                callback_name: "create_message"
+```

__init__.py

@@ -1,17 +1,89 @@
 from opsdroid.skill import Skill
 from opsdroid.matchers import match_rasanlu, match_parse
 import random
+import importlib
 
 class Respond(Skill):
+    # Constructor gets configs for the skill and applies the decorator
+    # to match using rasa.
     def __init__(self, opsdroid, config):
         super().__init__(opsdroid, config)
-        self.intent = self.config.get("intent")
+        self._get_configs()
         self.respond = match_rasanlu(self.intent)(self.respond)
 
+    # This utility function will grab the relevent config variables
+    # for this skill to function.
+    def _get_configs(self):
+        # The intent to match.
+        self.intent = self.config.get("intent")
+        # Possible responses given a match.
+        self.response_options = self.config.get("response_options")
+        # A bool to check if we want to use entities in our response.
+        self.use_entities = self.config.get("use_entities", False)
+        # Fallback responses if we do not get the required entities
+        # from Rasa.
+        self.fallback_response_options = self.config.get("fallback_response_options", 
+                ["Sorry, Rasa did not extract sufficient entities to fulfil your request"])
+        # An optional callback used for more advanced responses. Such
+        # functions should simply return the final response string and
+        # will be given the message and extracted entities (if enabled).
+        self.callback_file = self.config.get("callback_file", None)
+        self.callback_name = self.config.get("callback_name", None)
+
+
+    # Extract the entities from the rasanlu response
+    def _get_entities(self, message):
+        # This includes data such as the extractor used, which we
+        # don't need.
+        raw_entities = message.rasanlu["entities"]
+        entities = {}
+        for entity in raw_entities:
+            entity_type = entity["entity"]
+            unique_entity_type = self._get_unique_key(entity_type, entities)
+            entity_value = entity["value"]
+            entities[unique_entity_type] = entity_value
+        return entities
+
+                
+    # Appends the smallest integer to the name of the key in order to
+    # create a new key that is not already in the dictionary.
+    def _get_unique_key(self, base_key, dictionary):
+        counter = 0
+        while True:
+            unique_key_attempt = base_key + str(counter)
+            if unique_key_attempt in dictionary:
+                counter += 1
+                continue
+            return unique_key_attempt
+
+
+    # Takes the response and attempts to replace all entities in it
+    # with the extracted entities. Failing that, we will fall back to a
+    # fallback response.
+    def _substitute_entities(self, response, entities):
+        try:
+            response = response.format(**entities)
+        except KeyError as e:
+            chosen_fallback = random.choice(self.fallback_response_options)
+            return chosen_fallback
+        return response
+
+
     # This line is to add properties to respond on declaration
     # as doing this in __init__ would cause an exception to be
     # raised. If you don't believe me, remove it!
     @match_parse("hgftgyhjknbvhgftyuihjkvhgftyuijkbjhgtyyuijhkjghfdtrytyihujkbvncgfxdtrytuyiuhkbjvbhcfdytryuhjkbvhcfdyrtuyiuhkbhj")
     async def respond(self, message):
-        response_options = self.config.get("response_options")
+        chosen_response = random.choice(self.response_options)
-        await message.respond(random.choice(response_options))
+        entities = None # In case we want callbacks without entities
+        if self.use_entities:
+            entities = self._get_entities(message)
+            chosen_response = self._substitute_entities(chosen_response, entities)
+        if self.callback_file != None and self.callback_name != None:
+            # The following three lines allow us to import the module using an absolute path dynamically
+            spec = importlib.util.spec_from_file_location("callback", self.callback_file)
+            callback_module = importlib.util.module_from_spec(spec)
+            spec.loader.exec_module(callback_module)
+            # Now we can get the response from the callback.
+            chosen_response = eval("callback_module." + self.callback_name + "(message, entities)")
+        await message.respond(chosen_response)

examples/weather_callback.py

@@ -0,0 +1,34 @@
+import requests
+
+def create_message(message, entities):
+    # Check that we have a city to check on.
+    if 'GPE0' not in entities.keys():
+        return "ERROR: No GPE detected. Please include a location in your weather request."
+    # Read the api key to authenticate ourselves with.
+    with open('.openweathermapkey', 'r') as f:
+        apikey = f.readline().strip('\n') # Note that we need to strip the trailing \n 
+    # Create the request using the location and key
+    owm_url = f"https://api.openweathermap.org/data/2.5/weather?q={entities['GPE0']}&appid={apikey}"
+    owm_resp = requests.get(owm_url)
+    # Handle error codes
+    if owm_resp.status_code == 401:
+        return "ERROR: Sorry, there was an error with the api key for open weather map"
+    elif owm_resp.status_code == 404:
+        return f"ERROR: Sorry, open weather map does not recognise {entities['GPE0']} as a valid city"
+    elif owm_resp.status_code == 429:
+        return "ERROR: Sorry, it seems that there have been too many requests to get weather recently. Please try again later."
+    elif owm_resp.status_code in [500, 502, 503, 504]:
+        return "ERROR: How on Earth did you manage to break things this badly! Let Damien know what shenanigans you pulled to make this message appear."
+    elif owm_resp.status_code != 200:
+        return f"ERROR: I got {owm_resp.status_code} as a response from open weather map"
+
+    # Reponse is JSON, so simply parse and use to create response
+    owm_content = owm_resp.json()
+    return f"""<p>Here is the weather in {entities['GPE0']}:</p>
+<ul>
+<li>Weather is described as {owm_content['weather'][0]['description']}.</li>
+<li>The temperature is {owm_content['main']['temp']}K but it feels like {owm_content['main']['feels_like']}K and has been between {owm_content['main']['temp_min']}K and {owm_content['main']['temp_max']}K today.</li>
+<li>The pressure is {owm_content['main']['pressure']}hPa.</li>
+<li>The humidity is {owm_content['main']['humidity']}%.</li>
+<li>The wind is travelling at {owm_content['wind']['speed']}m/s at a heading of {owm_content['wind']['deg']}.</li>
+</ul>"""