This blog post introduces you to Azure Content Moderator and shows how to integrate it to blog post commenting process as a text moderator. Azure Content Moderator gives recommendations during the process whether comment should be reviewed or not. Review is recommended by Azure Content Moderator if comment contains offensive words or detects profanity. Commenting system used in this example doesn't support automatic approval of the comments so instead of that email notification with review recommendation is generated.
What is Azure Content Moderator?
Let's cover first some details about Azure Content Moderator.
Azure Content Moderator is an AI service that lets you handle content that is potentially offensive, risky, or otherwise undesirable. It includes the AI-powered content moderation service which scans text, image, and videos and applies content flags automatically. Source
Azure Content Moderator enables automation of processes which require reviewing the content before publishing. Content Moderator can review text and as well as image contents. You can also create own custom lists to screen against with Terms API. Current language support list is available in here.
This blog post concentrates to show how text moderation can be integrated comment approval process to give review recommendations for blog post comments.
Comment handling in this blog
Blog post commenting in this blog is handled with a service called Cusdis which is a lightweight, privacy-first and open-source comment system. Cusdis is integrated to this blog via JS SDK which requires just adding commenting widget element and reference to Cusdis JS SDK to the page.
Cusdis also supports Webhook to deliver events when new comment is added. Webhook is a key component which enables to add Azure Content Moderator to be a part of the process.
Old comment approval flow
By default Cusdis sends email notification about new comments. Comment notification email contains a link with a authentication token which enables comment approval without login.
Updated comment approval flow
This new updated flow relays on Webhook to deliver events about new comments. Azure Logic App receives and orchestrates handling of new comment and moderates the content with Azure Content Moderator. Finally comment with moderation results are persisted to table storage and email notification is sent to moderator.
Required Azure infrastructure
Resource group
az group create --name rg-blog --location westeuropeAzure Content Moderator
Azure Content Moderator is responsible to provide review recommendation about the comment.
az cognitiveservices account create --name cog-comment-moderation --resource-group rg-blog --kind ContentModerator --sku S0 --location westeurope --yesStorage Account
Storage Account and especially Table Storage is used to persist all comments and review recommendation results provided by Azure Content Moderator.
az storage account create -n stblogcomments -g rg-blog -l westeurope --sku Standard_LRSAzure Communication Service
Azure Communication Service is used in this sample to send email notification to moderator.
Note! You need to create also Azure Email Communication Service which is currently in a preview mode (01/2023) and available only from United States. Azure Communication Service must be located also to United States that you can use it for email communication.
az communication create --name comm-email --location "Global" --data-location "United States" --resource-group rg-blogLogic Apps
Logic App provides the Webhook endpoint and orchestrates the handling of comment.
az logic workflow create --resource-group rg-blog --location westeurope --name "logic-comment-webhook" --definition "definition.json"Technical implementation
Logic App
Logic App orchestrates the comment approval flow from receiving an event, requesting content moderation from Azure Content Moderator, persisting the comment and results to Table Storage and sending an email to moderator.
Step 1: When a HTTP request is received
This action creates HTTP POST endpoint which receives comment events from Cusdis. You can find new comment event schema documentation from here.
Step 2: Request moderation from Azure Content Moderator
Request moderation action detects offensive & profanity words and match against custom and shared block lists.
Output of this action (API call) is a JSON object which has classification in three categories. These categories are determined as follows according to Microsoft documentation:
Category1 refers to potential presence of language that may be considered sexually explicit or adult in certain situations.
Category2 refers to potential presence of language that may be considered sexually suggestive or mature in certain situations.
Category3 refers to potential presence of language that may be considered offensive in certain situations.
Category score is between 0 and 1. The higher the score, the higher the model is predicting that the category may be applicable. This feature relies on a statistical model rather than manually coded outcomes. Microsoft recommends testing with your own content to determine how each category aligns to your requirements.
ReviewRecommended is either true or false depending on the internal score thresholds.
{
"body": {
"OriginalText": "Thank you for sharing this. Very useful content.",
"NormalizedText": "Thank you sharing . useful content.",
"Misrepresentation": null,
"Classification": {
"ReviewRecommended": false,
"Category1": {
"Score": 0.057542379945516586
},
"Category2": {
"Score": 0.17334049940109253
},
"Category3": {
"Score": 0.04330739378929138
}
},
"Language": "eng",
"Terms": null,
"Status": {
"Code": 3000,
"Description": "OK",
"Exception": null
},
"TrackingId": "6c541d65-ccd6-4a0c-a157-cb6629323250"
}
}Step 3: Parse Content Moderation Result
This action creates user-friendly tokens from JavaScript Object Notation (JSON) object properties so you can easily use those properties in the workflow.
Step 4: Initialize Table Storage Entity
This action initializes the comment entity which has information from raw event and results of Azure Content Moderator.
Step 5: Persists Data to Table Storage
Comment entity is persisted to the table storage via this action.
Step 6: Initialize Quickchart data
This action initializes a variable which contains Quickchart JSON object as a string. Quickchart is used to generate a diagram based on Azure Content Moderator classification results. Later this chart is embedded to notification email.
Step 7: Encode Quickchart content
This variable creates an HTML image tag which refers to Quickchart API service to generate chart image. Querystring parameter "C" contains Quickchart JSON object as a URI encoded format.
Step 8: Create Approval Link
Like said earlier Cusdis Webhook event contains a link with a authentication token which enables comment approval without login. This variable creates an HTML link with that approval link.
Step 9: Send notification email
This Send Email action of Azure Communication Service gathers all required information to body of email and delivers email notification to moderator.
Final email notification looks like this. It contains original comment, review recommendation provided by Azure Content Moderator and chart provided by Quickchart about classification details of moderation.
Summary
My initial plan was to test Azure Content Moderator API via Azure Functions but quickly I noticed that same API capabilities are also available in Logic App with pre-built actions. Creating this orchestration of the approval flow was really easy and straightforward with Logic Apps. It was also interested to investigate a bit about Azure Communication Service capabilities.
Hopefully, some day I can automate this so that moderator interaction is required only if Azure Content Moderator suggests manual review. Otherwise comment is approved automatically via API (which is not currently possible).
Logic App definition file is available in Github.
Comments