OPSWAT uses NetSuite as its CRM and ERP system. Our engineering team has been integrating with NetSuite through its various APIs, to transfer information in and out of the system. As our product family offering has grown, we have needed to constantly maintain and update this information flow by updating the NetSuite scripts. This process has been prone to human error and very challenging to debug as deep NetSuite knowledge is needed in order to trace down sources of issues. Faced with these issues, our engineering team began looking for a solution to externalize the logic for the routing of information from within the NetSuite scripts. In addition, we wanted to make sure that we did not have tight coupling between our systems, so that if a particular system is unavailable, we would still be able to update records in NetSuite without compromising data consistency.
We wanted a solution that would enable us to easily configure any endpoint to receive updates from NetSuite. Since any number of systems could be interested in receiving the updates, a publish-subscribe model was suitable. A solution that presented itself to our team was Amazon SNS, a fully managed publish-subscribe service with support for a variety of endpoints including HTTP/HTTPS and email. The SNS solution facilitated authentication, authorization, integrity verification and the ability to dynamically configure the information flow out of NetSuite. A useful feature of SNS for HTTP/HTTPS is the retry policy, which allows us to specify the number of attempts and delays between them. Without such feature, we would end up with tightly coupled systems, which would create a domino effect when one of the systems is having availability issues. Another out-of-the-box feature of Amazon SNS is CloudWatch reporting, so our DevOps team is able to monitor the information flow and be alerted to issues immediately.
To validate our decision to use Amazon SNS, we decided to do a proof-of-concept implementation. During that process, we had to overcome various technical challenges to get NetSuite and Amazon SNS to work together. It was relatively straightforward to get the solution working. The only step that needed time investment was digitally signing the SNS message using JavaScript, and for that we used the open-source library CryptoJS.
Our proof of concept worked out well, and we have since then rolled it out to production. We decided to take the lessons we learned during the implementation process and produced a step-by-step guide for other engineers who are looking to implement a similar solution.
Prerequisite Knowledge Needed
First of all, you need to have basic knowledge about the following:
- SNS and REST API:
- NetSuite and SuiteScript API:
Step By Step Guide
1. Create a topic in Amazon SNS service
2. Write suite script to post message to SNS
Download 2 library files, hmac-md5.js and hmac-sha256.js, from crypto-js project and write publishSNS.js script.
Sample Script
<span style="color: #008800; font-weight: bold">var</span> access_key <span style="color: #333333">=</span> <span style="background-color: #fff0f0">''</span>;<span style="color: #008800; font-weight: bold">var</span> secret_key <span style="color: #333333">=</span> <span style="background-color: #fff0f0">''</span>;<span style="color: #008800; font-weight: bold">var</span> region <span style="color: #333333">=</span> <span style="background-color: #fff0f0">'us-east-1'</span>;<span style="color: #008800; font-weight: bold">var</span> endpoint <span style="color: #333333">=</span> <span style="background-color: #fff0f0">'https://sns.us-east-1.amazonaws.com/'</span>;<span style="color: #008800; font-weight: bold">var</span> host <span style="color: #333333">=</span> <span style="background-color: #fff0f0">'sns.us-east-1.amazonaws.com'</span>;<span style="color: #008800; font-weight: bold">var</span> method <span style="color: #333333">=</span> <span style="background-color: #fff0f0">'POST'</span>;<span style="color: #008800; font-weight: bold">var</span> service <span style="color: #333333">=</span> <span style="background-color: #fff0f0">'sns'</span>;<span style="color: #008800; font-weight: bold">var</span> targetArn <span style="color: #333333">=</span> <span style="background-color: #fff0f0">''</span>; <span style="color: #008800; font-weight: bold">function</span> setConfiguration(){ access_key <span style="color: #333333">=</span> nlapiGetContext().getSetting(<span style="background-color: #fff0f0">'SCRIPT'</span>,<span style="background-color: #fff0f0">'custscript_aws_access_key'</span>); secret_key <span style="color: #333333">=</span> nlapiGetContext().getSetting(<span style="background-color: #fff0f0">'SCRIPT'</span>,<span style="background-color: #fff0f0">'custscript_aws_secret_key'</span>); targetArn <span style="color: #333333">=</span> nlapiGetContext().getSetting(<span style="background-color: #fff0f0">'SCRIPT'</span>,<span style="background-color: #fff0f0">'custscript_sns_target_arn'</span>);} <span style="color: #008800; font-weight: bold">function</span> sign(key, msg){ <span style="color: #008800; font-weight: bold">return</span> CryptoJS.HmacSHA256(msg, key);} <span style="color: #008800; font-weight: bold">function</span> getSignatureKey(key, dateStamp, regionName, serviceName){ kDate <span style="color: #333333">=</span> sign(<span style="background-color: #fff0f0">'AWS4'</span> <span style="color: #333333">+</span> key, dateStamp); kRegion <span style="color: #333333">=</span> sign(kDate, regionName); kService <span style="color: #333333">=</span> sign(kRegion, serviceName); kSigning <span style="color: #333333">=</span> sign(kService, <span style="background-color: #fff0f0">'aws4_request'</span>); <span style="color: #008800; font-weight: bold">return</span> kSigning;} <span style="color: #008800; font-weight: bold">function</span> editAction(){ <span style="color: #888888">// set values for global variables, take parameters from netsuite configure</span> setConfiguration(); <span style="color: #888888">// Create a date for headers and the credential string</span> <span style="color: #008800; font-weight: bold">var</span> amzdate <span style="color: #333333">=</span> <span style="color: #008800; font-weight: bold">new</span> <span style="color: #007020">Date</span>().toISOString(); amzdate <span style="color: #333333">=</span> amzdate.split(<span style="background-color: #fff0f0">"."</span>)[<span style="color: #0000DD; font-weight: bold">0</span>]<span style="color: #333333">+</span><span style="background-color: #fff0f0">"Z"</span>; amzdate <span style="color: #333333">=</span> amzdate.replace(<span style="color: #000000; background-color: #fff0ff">/-/g</span>,<span style="background-color: #fff0f0">""</span>).replace(<span style="color: #000000; background-color: #fff0ff">/:/g</span>,<span style="background-color: #fff0f0">""</span>); <span style="color: #008800; font-weight: bold">var</span> datestamp <span style="color: #333333">=</span> amzdate.split(<span style="background-color: #fff0f0">"T"</span>)[<span style="color: #0000DD; font-weight: bold">0</span>]; <span style="color: #888888">// Sample message</span> <span style="color: #008800; font-weight: bold">var</span> request_parameters <span style="color: #333333">=</span> <span style="background-color: #fff0f0">"Action=Publish&Message="</span> <span style="color: #333333">+</span> <span style="color: #007020">encodeURIComponent</span>(<span style="background-color: #fff0f0">"This is test message"</span>) <span style="color: #333333">+</span> <span style="background-color: #fff0f0">"&Subject="</span> <span style="color: #333333">+</span> <span style="color: #007020">encodeURIComponent</span>(<span style="background-color: #fff0f0">"Title"</span>) <span style="color: #333333">+</span> <span style="background-color: #fff0f0">"&TargetArn="</span> <span style="color: #333333">+</span> escape(targetArn) <span style="color: #333333">+</span> <span style="background-color: #fff0f0">"&Version=2010-03-31"</span>; <span style="color: #888888">// ************* TASK 1: CREATE A CANONICAL REQUEST *************</span> <span style="color: #888888">// http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html</span> <span style="color: #888888">// Step 1 is to define the verb (GET, POST, etc.)--already done.</span> <span style="color: #888888">// Step 2: Create canonical URI--the part of the URI from domain to query</span> <span style="color: #888888">// string (use '/' if no path)</span> <span style="color: #008800; font-weight: bold">var</span> canonical_uri <span style="color: #333333">=</span> <span style="background-color: #fff0f0">'/'</span>; <span style="color: #888888">// Step 3: Create the canonical query string. In this example (a GET request),</span> <span style="color: #888888">// request parameters are in the query string. Query string values must</span> <span style="color: #888888">// be URL-encoded (space=%20). The parameters must be sorted by name.</span> <span style="color: #888888">// For this example, the query string is pre-formatted in the request_parameters variable.</span> <span style="color: #008800; font-weight: bold">var</span> canonical_querystring <span style="color: #333333">=</span> request_parameters; <span style="color: #888888">// Step 4: Create the canonical headers and signed headers. Header names</span> <span style="color: #888888">// and value must be trimmed and lowercase, and sorted in ASCII order.</span> <span style="color: #888888">// Note that there is a trailing
.</span> <span style="color: #008800; font-weight: bold">var</span> canonical_headers <span style="color: #333333">=</span> <span style="background-color: #fff0f0">'host:'</span> <span style="color: #333333">+</span> host <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'
'</span> <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'x-amz-date:'</span> <span style="color: #333333">+</span> amzdate <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'
'</span>; <span style="color: #888888">// Step 5: Create the list of signed headers. This lists the headers</span> <span style="color: #888888">// in the canonical_headers list, delimited with ";" and in alpha order.</span> <span style="color: #888888">// Note: The request can include any headers; canonical_headers and</span> <span style="color: #888888">// signed_headers lists those that you want to be included in the</span> <span style="color: #888888">// hash of the request. "Host" and "x-amz-date" are always required.</span> <span style="color: #008800; font-weight: bold">var</span> signed_headers <span style="color: #333333">=</span> <span style="background-color: #fff0f0">'host;x-amz-date'</span>; <span style="color: #888888">// Step 6: Create payload hash (hash of the request body content). For GET</span> <span style="color: #888888">// requests, the payload is an empty string ("").</span> <span style="color: #008800; font-weight: bold">var</span> payload_hash <span style="color: #333333">=</span> <span style="background-color: #fff0f0">'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'</span>; <span style="color: #888888">// Step 7: Combine elements to create create canonical request</span> <span style="color: #008800; font-weight: bold">var</span> canonical_request <span style="color: #333333">=</span> method <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'
'</span> <span style="color: #333333">+</span> canonical_uri <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'
'</span> <span style="color: #333333">+</span> canonical_querystring <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'
'</span> <span style="color: #333333">+</span> canonical_headers <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'
'</span> <span style="color: #333333">+</span> signed_headers <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'
'</span> <span style="color: #333333">+</span> payload_hash; <span style="color: #888888">// ************* TASK 2: CREATE THE STRING TO SIGN*************</span> <span style="color: #888888">// Match the algorithm to the hashing algorithm you use, either SHA-1 or</span> <span style="color: #888888">// SHA-256 (recommended)</span> <span style="color: #008800; font-weight: bold">var</span> algorithm <span style="color: #333333">=</span> <span style="background-color: #fff0f0">'AWS4-HMAC-SHA256'</span>; <span style="color: #008800; font-weight: bold">var</span> credential_scope <span style="color: #333333">=</span> datestamp <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'/'</span> <span style="color: #333333">+</span> region <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'/'</span> <span style="color: #333333">+</span> service <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'/'</span> <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'aws4_request'</span>; <span style="color: #008800; font-weight: bold">var</span> string_to_sign <span style="color: #333333">=</span> algorithm <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'
'</span> <span style="color: #333333">+</span> amzdate <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'
'</span> <span style="color: #333333">+</span> credential_scope <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'
'</span> <span style="color: #333333">+</span> CryptoJS.SHA256(canonical_request); <span style="color: #888888">// ************* TASK 3: CALCULATE THE SIGNATURE *************</span> <span style="color: #888888">// Create the signing key using the function defined above.</span> <span style="color: #008800; font-weight: bold">var</span> signing_key <span style="color: #333333">=</span> getSignatureKey(secret_key, datestamp, region, service); <span style="color: #888888">// Sign the string_to_sign using the signing_key</span> <span style="color: #008800; font-weight: bold">var</span> signature <span style="color: #333333">=</span> CryptoJS.HmacSHA256(string_to_sign, signing_key); <span style="color: #008800; font-weight: bold">var</span> authorization_header <span style="color: #333333">=</span> algorithm <span style="color: #333333">+</span> <span style="background-color: #fff0f0">' '</span> <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'Credential='</span> <span style="color: #333333">+</span> access_key <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'/'</span> <span style="color: #333333">+</span> credential_scope <span style="color: #333333">+</span> <span style="background-color: #fff0f0">', '</span> <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'SignedHeaders='</span> <span style="color: #333333">+</span> signed_headers <span style="color: #333333">+</span> <span style="background-color: #fff0f0">', '</span> <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'Signature='</span> <span style="color: #333333">+</span> signature; <span style="color: #888888">// The request can include any headers, but MUST include "host", "x-amz-date",</span> <span style="color: #888888">// and (for this scenario) "Authorization". "host" and "x-amz-date" must</span> <span style="color: #888888">// be included in the canonical_headers and signed_headers, as noted</span> <span style="color: #888888">// earlier. Order here is not significant.</span> <span style="color: #888888">// Python note: The 'host' header is added automatically by the Python 'requests' library.</span> <span style="color: #008800; font-weight: bold">var</span> headers <span style="color: #333333">=</span> {<span style="background-color: #fff0f0">'x-amz-date'</span><span style="color: #333333">:</span>amzdate, <span style="background-color: #fff0f0">'Authorization'</span><span style="color: #333333">:</span>authorization_header}; <span style="color: #888888">// ************* SEND THE REQUEST *************</span> <span style="color: #008800; font-weight: bold">var</span> request_url <span style="color: #333333">=</span> endpoint <span style="color: #333333">+</span> <span style="background-color: #fff0f0">'?'</span> <span style="color: #333333">+</span> canonical_querystring; <span style="color: #008800; font-weight: bold">var</span> response <span style="color: #333333">=</span> nlapiRequestURL(request_url, <span style="background-color: #fff0f0">""</span>, headers); <span style="color: #008800; font-weight: bold">if</span>(response.getCode() <span style="color: #333333">===</span> <span style="color: #0000DD; font-weight: bold">200</span>){ nlapiLogExecution(<span style="background-color: #fff0f0">'DEBUG'</span>,<span style="background-color: #fff0f0">'test'</span>,<span style="background-color: #fff0f0">'Published to SNS'</span>); } <span style="color: #008800; font-weight: bold">else</span>{ nlapiLogExecution(<span style="background-color: #fff0f0">'DEBUG'</span>,<span style="background-color: #fff0f0">'test'</span>,<span style="background-color: #fff0f0">'Failed to publish to SNS '</span> <span style="color: #333333">+</span> response.getCode()); }}3. Configure SNS to execute this script when user make change on records
- Create a folder (e.g., PublishSNS) under SuiteScripts (Menu Documents > Files > SuiteScripts) and upload 3 js script files from the above steps.
- Create the script record
- Select Type (Menu Setup > Other setup > Script, click "New script" and choose "User Event").
- Define the script
- NAME: PublishSNS
- SCRIPT FILE: select publishSNS.js
- After Submit function: editAction
- Add libraries: choose 2 library files: hmac-md5.js, hmac-sha256.js
- Leave empty for BEFORE LOAD FUNCTION and AFTER SUBMIT FUNCTION
- Create 3 parameters:
Label: AWS Access Key
ID: _aws_access_key
TYPE: Free-form text
Do similar steps with AWS Secret Key (_aws_secret_Key), SNS Target Arn (_sns_target_arn).
- Choose "save and deploy"
- Select Type (Menu Setup > Other setup > Script, click "New script" and choose "User Event").
- Deploy script
- Status: Release
- Apply to: Contact
- Event type: Empty for all even
- Audience:
- ROLES: Select All
- PARTNER: Select All
- EMPLOYEES: Select All
- Parameters:
Fill in your keys and Target ARN info.
How To Verify
- Make any change on contact record and check "Execution Log".
- If you have already subscribed you should get notification message from Amazon SNS.
From Deployment to Production
We tested out the notification process in a NetSuite sandbox initially, and the results were as expected. There was no performance impact on the user experience and notifications were triggered whenever a user manually updated a record as well as when other scripts updated records. An easy way to keep an eye on the flow of data is to subscribe via email to the SNS topic so that there is another copy of all the messages sent to the SNS topic. CloudWatch monitoring was also helpful, as we had setup alerts for failed deliveries that proved helpful during the test phase. We changed the default settings on SNS HTTP retry from 3 times to 10 times, just to be safe, and have not had any issues with failed message delivery in production.
Overall, the implementation process was a low effort exercise with good payback. With the SNS topic pub-sub mechanism, our internal applications can now listen in for NetSuite updates. This saves us time, as we won't need to go through a release cycle every time a new application needs to receive NetSuite updates. All we need to do now is add a new SNS topic subscriber, and we are ready to roll!
