February 22, 2018 / by

Hello Microsoft Teams, this is NiFi calling

We all love to chat! And we love when our machines join the chat… In this article we see how you can easily integrate Apache NiFi with Microsoft Teams

In case you haven’t heard about, MSFT Teams is Redmond’s response to Slack, HipChat or one of their open-source alternatives like Rocket.chat. To be honest, it is not a particularly bad tool, however, being one of the new kids on the block, its ecosystems is still limited.

Perhaps a perfect example of this, is the very fact that even NiFi lacks a PutTeams processor (while still offering a PutSlack processor)…

However, at Fluenda we strongly believe that as long as there’s a stable interface, NiFi will always be able to pipe the data between A and B.

So without much further talk, lets integrate Apache NiFi with Microsoft Teams.

Prerequisites

To complete this task you will need to fulfil the following prerequisites:

• A working deployment of Apache NiFi;
• An Office 365 license with access to Microsoft Teams (e.g. E3 trial);

Getting started

Before we integrate it is important to note that MSFT Teams has two different types of webhooks:

• Outgoing webhooks
• Incoming webhooks (aka Incoming Webhook Connector)

The main difference is that the Outgoing webhook is used to receive and reply to messages from users within Team, while the second type allows you to freely post into a channel.

In this tutorial we address the Incoming WebHooks scenario.

Setting up the Incoming Webhook

Step 1. Navigate to the team you intent to post to and after clicking the “…” next to the channel, select Manage Team from the pop-up menu.

Step 2. Navigate to the Apps tab and click “Go to store”

Step 3. Search for the Connector “Incoming Webhook” and install it to the Team of your choice.

Step 4. The connector will prompt you to setup the webhook. Choose a name, pick an icon and click create.

Step 5. Upon creation Teams will provide you with an URL. Write it down and click Done.

Upon returning to the channel you should see a notification the deployment of the connector was successful.

Testing the basic setup

Test the connector by calling the webhook URL with curl:

curl -H "Content-Type: application/json" \
-d "{\"text\": \"Hello World from curl\"}" <replace_with_the_url_you_received>

and you should see “Hello World from curl” appearing in the chat.

Integrating NiFi

But how to integrate this into NiFi?

It couldn’t be simpler!

All you need to do is to add a particularly formatted flowfile into your NiFi flow. Good candidates are MonitorActivity, GenerateFlowfile and most of the low volume content generators such as reporting tasks, etc.

No matter how you generate the flowfile, the content you post should look like this:

{
    "@type": "MessageCard",
    "@context": "http://schema.org/extensions",
    "summary": "A sample message by NiFi-a-Bot",
    "title": "The Aussie NiFi Ninja hint of the day",
    "sections": [
        {
            "title": "Sometimes you just need to read the API documentation...",
            "facts": [
                {
                    "name": "Tags",
                    "value": "Apache NiFi, Integration, DevOps, Automation"
                },
                {
                    "name": "Published on",
                    "value": "21 Feb 2018"
                }
            ]
        }
    ]
}

We generate flowfile and in addition to the content above we also add an attribute we call test. We will use it later…

With the content properly formatted all we need is to configure InvokeHttp processor to POST the content to the URL provided by the Teams connector setup page

And voila! The channel gets updated with the message we generated.

But what if you wanted to post dynamically crafter messages? Well, it happens that the content payload is just JSON… and nothing better to edit JSON than ExecuteScri… I mean! JoltTransformJson!! You got to love to hate Jolt Transforms!

So…

Add the JoltTransformJson processor to the canvas, route your content through it and finally configure the processor with the following transformation:

[{
	"operation": "shift",
	"spec": {
		"sections": {
			"*": {
				"title": {
					"#${test}": "sections[].title"
				},
				"facts": "sections[&1].facts"
			}
		},
		"*": "&"
	}
}]

The result will look like this:

Press start and note how the new posts reflect the modified content:

Closing remarks

As you can see, NiFi is an incredibly powerful platform.

From one side, it that can be used to handle vast amounts of data, from logs to data dumps with equal ease.

However, NiFi can also be used to integrate different parts of a DevOps ecosystem, acting as glue between disparate systems.

Why is this important?

Imagine for a second that your team leveraged batch oriented reporting tools such as Splunk Analytics for Hadoop…

What it you could use NiFi to periodically query the Search Head for results, taking actions once results are made available?

More importantly: What if this tool also acted as the backbone of your data pipeline, reducing the number of tools you have to manage?

In Fluenda’s opinion, the ability to use the same tool to process and orchestrate data operations is one of NiFi’s best kept secrets.

Nonetheless, as the post above demonstrates, using NiFi to safely link your systems with Microsoft Teams is an extremely easy task.

In the next post we will see how NiFi can be used to automatically react to interactions with other Teams users. Stay tuned!