Gettings started using the Inversoft Profanity WebService
This document covers how to get started using the Inversoft Profanity WebService. In order to illustrated simple usages of the WebService this guide uses Java and Ruby code snippets, however, any language that provides HTTP and XML libraries should be able to use the WebService.
Creating a Profanity WebService request
The first step in using the Inversoft Profanity WebService is to create an profanityRequest that is a well formed XML document which will be sent to the WebService. This XML document must conform to the profanityRequest XML Schema document, which is located at:
http://www.inversoft.com/schemas/profanity-1.0/webservice/request.xsd
This schema controls the entire operation of the WebService including the type of operation that the WebService should perform. For this guide will will assume that a REPLACE operation is being performed. Documentation on the FIND operation can be found in the XML Schema documents themselves.
Let's assume that a user has submitted some content such as:
I can't understand why that dumb ass would do that.
We construct a profanityRequest XML document to send this content to the WebService and ask the WebService to replace all profanity in the content with the * character. This is how the profanityRequest would look:
<profanityRequest xmlns="http://www.inversoft.com/schemas/profanity-1.0"
tolerance="5" replacementCharacter="*" profanityTypes="Swear,Slang"
locale="en" filterType="STANDARD" operation="REPLACE">
<authentication username="your-username" password="your-password"/>
<text><![[CDATA[I can't understand why that dumb ass would do that.]]><</text>
</profanityRequest>
Listing 1: profanityRequest XML document
There are a few things to notice about the profanityRequest XML document from listing 1. First, there are 6 attributes of the root element. They each have the following meanings and all are required for a REPLACE operation.
ProfanityRequest root element attributes
The tolerance attribute controls how strict or lenient the WebService should be. This attribute informs the WebService that it should only check the text block for profanity words whose rating is equal to or greater than the tolerance. Each profanity word that the WebService searches for and replaces has a rating. The list of profanity words that the WebService searches for and replaces can be purchased from Inversoft. The product name of this list is the Inversoft Profanity Database. You can also email Inversoft to inquiry about the ratings for specific profanity words at info@inversoft.com.
The replacementCharacter is only used if the WebService is being called in REPLACE mode using the operation attribute. If the operation attribute is REPLACE than this attribute must be specified. This character is used by the WebService to replace profanity in the text block. For example, if the WebService is passed "This fucking website sucks" and a replacementCharacter of '*' it will return the text "This ******* website sucks". If the WebService is called in FIND mode, this attribute is ignored.
The profanityTypes attribute is a space separated list of profanity types to find or replace. This list uses the enumerated values of the profanityType simpleType. If you want to have the WebService filter Slang, Swear and Religious words from the text block than this attribute should be set to a value of "Slang Swear Religious".
The locale parameter informs the WebService what language the text block is written in. Currently, the Inversoft Profanity WebService only works with American English, but in the future other languages will be supported. This attribute is a combination of a language and country code and the documentation for locale type in the XML Schema document contains additional information. Currently, this attribute is ignored and the WebService only finds/replaces American English profanity.
The filterType parameter is used to tune the performance of the WebService. There are currently two settings, FAST and STANDARD. The FAST filterType informs the WebService that it should work as quickly as possible. This however could cause complex and tricky instances of profanity to be missed. The STANDARD filterType informs the WebService to work in standard mode and find as much profanity as possible. The STANDARD filterType does cause the WebService to run slower because it most make a more exhaustive scan of the text block. However, the WebService is fully tuned and often medium sized text blocks of around 200-400 words can be handled by the WebService running with the STANDARD filterType in less than 100 miliseconds. This does not include network latency.
The operation attribute tells the WebService what type of operation to perform. There are currently two operations, FIND and REPLACE. The result of each operation is different and controls the results in the response XML document that the WebService returns. The FIND operation informs the WebService to find all instances of profanity in the text block and a list of matches is returned. These include the word that was matched and the location in the text (see the profanityResponse XML Schema documentation for more information). The REPLACE operation informs the WebService to replace all profanity in the text block with the replacementCharacter and return the new text.
ProfanityRequest authentication
The profanityRequest XML root element contains your authentication information in the authentication element. This element has two attributes for the authentication information, username and password. Both of these are required and must be the credentials for an active account.
ProfanityRequest text
Lastly, the profanityRequest XML document contains the text being filtered by the WebService in the text element. This element must contain the text within a CDATA block. This prevents improper handling of the text if it contains XML or similar type of content.
Contacting the WebService
The WebService is accessible via the URL https://profanity.inversoft.com:7448/webservice. In order to contact the WebService the profanityRequest XML document must be set as the HTTP body of the HTTP request. Here is some sample code in Java and Ruby for accomplishing this:
try {
String request =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
"<profanityRequest xmlns=\"http://www.inversoft.com/schemas/profanity-1.0\"" +
" tolerance=\"5\" replacementCharacter=\"*\" profanityTypes=\"Swear Slang\"" +
" locale=\"en\" filterType=\"STANDARD\" operation=\"REPLACE\">" +
"<authentication username=\"your-username\" password=\"your-password\"/>" +
"<text><![CDATA[I can't understand why that dumb ass would do that.]]></text>" +
"</profanityRequest>";
URL url = new URL("https://profanity.inversoft.com/webservice");
URLConnection urlConnection = url.openConnection();
// Setup for read and write
urlConnection.setDoInput(true);
urlConnection.setDoOutput(true);
urlConnection.connect();
// Write the request
OutputStream outputStream = urlConnection.getOutputStream();
outputStream.write(request.getBytes());
outputStream.flush();
} catch (Exception e) {
System.err.println("Unable to contact WebService");
e.printStackTrace();
}
Listing 2: Sample Java request code
require 'net/http'
begin
request = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
"<profanityRequest xmlns=\"http://www.inversoft.com/schemas/profanity-1.0\"" +
" tolerance=\"5\" replacementCharacter=\"*\" profanityTypes=\"Swear Slang\"" +
" locale=\"en\" filterType=\"STANDARD\" operation=\"REPLACE\">" +
"<authentication username=\"your-username\" password=\"your-password\"/>" +
"<text><![CDATA[I can't understand why that dumb ass would do that.]]></text>" +
"</profanityRequest>"
http = Net::HTTP.new("localhost", 8080)
response = http.post("/webservice", request)
rescue => e
$stderr.puts "Unable to contact WebService"
$stderr.puts e.to_str
$stderr.puts e.backtrace.join("\n")
end
Listing 3: Sample Ruby request code
Both of these code examples open up a socket to the server profanity.inversoft.com and then send an HTTP post message to the server with the profanityRequest XML document in the body. Of course, this is fairly brute force, but most implementations will use similar approachs to connecting to the WebService.
Reading the result
The last step in contacting the Inversoft Profanity WebService is to read the response. The response is always a well formed XML document that conforms the the XML Schema located at http://www.inversoft.com/schemas/profanity-1.0/webservice/response.xsd . A profanityResponse XML document can be parsed using any XML compliant parsing library.
The profanityResponse XML document will contain either a successful result or a failure. If the response is invalid, it will contain a XML document like this:
<profanityResponse xmlns="http://www.inversoft.com/schemas/profanity-1.0">
<invalid><error code="1">Invalid credentials</error></invalid>
</profanityResponse>
Listing 4: Invalid profanityResponse XML document
This response contains the profanityResponse element and the invalid element inside that. The invalid element contains only a single element called error, which contains the error code and a detailed error message.
If the profanityResponse XML document contains a successful result it will look like this:
<profanityResponse xmlns="http://www.inversoft.com/schemas/profanity-1.0">
<valid>
...
</valid>
</profanityResponse>
Listing 5: Valid profanityResponse XML document
This response contains the valid element rather than the invalid element to denote a successful response. This element has different content based on the operation sent to the WebService in the profanityRequest. Since we are invoking the REPLACE operation our successful response will contain a text element with all of the profanity replaced. Here is an example of a successful profanityResponse from a REPLACE operation.
<profanityResponse xmlns="http://www.inversoft.com/schemas/profanity-1.0">
<valid>
<text><![[CDATA[I can't understand why that dumb *** would do that.]]></text>
</valid>
</profanityResponse>
Listing 6: Valid profanityResponse with text
This contains the original text sent to the WebService with the word ass replaced. Recall that the replacement character is controlled by the replacementCharacter attribute in the profanityRequest.
The last thing to do is to read the response from the HTTP response body. Here is some sample code showing how to do this. These samples assume that an external library will be parsing the XML result.
try {
String request =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
"<profanityRequest xmlns=\"http://www.inversoft.com/schemas/profanity-1.0\"" +
" tolerance=\"5\" replacementCharacter=\"*\" profanityTypes=\"Swear Slang\"" +
" locale=\"en\" filterType=\"STANDARD\" operation=\"REPLACE\">" +
"<text><![CDATA[I can't understand why that dumb ass would do that.]]></text>" +
"</profanityRequest>";
URL url = new URL("http://127.0.0.1:8080/webservice");
URLConnection urlConnection = url.openConnection();
// Setup for read and write
urlConnection.setDoInput(true);
urlConnection.setDoOutput(true);
urlConnection.connect();
// Write the request
OutputStream outputStream = urlConnection.getOutputStream();
outputStream.write(request.getBytes());
outputStream.flush();
// Read the response
InputStream inputStream = urlConnection.getInputStream();
DocumentBuilder documentBuilder = DocumentBuilderFactory.newInstance().newDocumentBuilder();
Document document = documentBuilder.parse(inputStream);
// See if it is valid or invalid
Element profanityResponse = document.getDocumentElement();
Element invalid = (Element) profanityResponse.getElementsByTagName("invalid").item(0);
if (invalid != null) {
// Invalid response!
Element error = (Element) invalid.getElementsByTagName("error").item(0);
System.err.println("WebService call failed due to [" + error.getTextContent() + "]");
} else {
// Valid response
Element valid = (Element) profanityResponse.getElementsByTagName("valid").item(0);
Element replace = (Element) valid.getElementsByTagName("replace").item(0);
Element text = (Element) replace.getElementsByTagName("text").item(0);
Node cdata = text.getFirstChild();
System.out.println("WebService call was successful. Cleaned message is [" +
cdata.getTextContent() + "]");
}
} catch (Exception e) {
System.err.println("Unable to contact WebService");
e.printStackTrace();
}
Listing 7: Sample Java response code
require 'net/http'
require 'rexml/document'
begin
request = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
"<profanityRequest xmlns=\"http://www.inversoft.com/schemas/profanity-1.0\""
" tolerance=\"5\" replacementCharacter=\"*\" profanityTypes=\"Swear Slang\"" +
" locale=\"en\" filterType=\"STANDARD\" operation=\"REPLACE\">" +
"<text></text>" +
"</profanityRequest>"
http = Net::HTTP.new("localhost", 8080)
response = http.post("/webservice", request)
document = REXML::Document.new(response.body)
profanityResponse = document.root
if profanityResponse.elements['invalid']
error = profanityResponse.elements['invalid'].elements['error']
puts "WebService call failed due to [" + error.text + "]"
else
message = profanityResponse.elements['valid'].elements['replace'].elements['text']
puts "WebService call was successful. Cleaned message is [" + message.text + "]"
end
rescue => e
$stderr.puts "Unable to contact WebService"
$stderr.puts e.to_str
$stderr.puts e.backtrace.join("\n")
end
Listing 8: Sample Ruby response code