Now and then customers attempt performance testing in sandbox mode and wonder why sandbox is so slow and why they get policy error about exceeding their TPS limit.
SANDBOX IS FOR TESTING THE CORRECTNESS OF YOUR APPLICATION – NOT FOR TESTING PERFORMANCE
Sandbox is a simulator for OpaaliAPI, it is especially useful for testing error cases which are hard to generate in Production Mode. Such as “Subscriber barred” or “Prepaid Subscriber with insufficient balance” – see the document API_Telia_OMA_Sandbox_REST.pdf in the Resources section of the Opaali Portal.
As a simulator its performance is not comparable to production so running any kind of performance tests in Sandbox is useless. And because it is a simulator, there is no connection to the actual telephone network.
NO, YOU CANNOT RECEIVE A MESSAGE SENT FROM SANDBOX INTO YOUR OWN PHONE
I’ll repeat: you cannot receive a message sent from sandbox into your own phone.
What good is it, then?
Once you have made it past the Opaali API equivalent of “Hello World” – your first application on a new platform – you should use the Sandbox to simulate how your application behaves in common error cases.
(what is the Opaali API equivalent to “Hello World” application? – it is of course the “400 Bad Request” application, which was also my first attempt at using the Opaali REST API :-) )
$ curl -i -d '{"outboundMessageRequest":{"address":["tel:+35812345001"],"senderAddress":"tel:+358000000000","outboundSMSTextMessage":{"message":"Hello World"}
,"senderName":"JPLa"}' "https://api.opaali.telia.fi/sandbox/messaging/v1/outbound/tel%3A%2B358000000000/requests" --header "Content-Type:application/json" --header "Authorization: Bearer $access_token"
HTTP/1.1 400 Bad Request
content-type: text/xml
Content-Length: 11
Bad Request
API_Telia_OMA_Sandbox_REST.pdf has various chapters listing Canned Responses for Predefined Parameter Values – lets try out some of them. You will get different responses depending on the address of the recipient.
Successful response (tel:+35812345001)
$ curl -k -d '{"outboundMessageRequest":{"address":["tel:+35812345001"],"senderAddress":"tel:+358000000000","outboundSMSTextMessage":{"message":"Hello World!"
},"senderName":"JPLa"}}' "https://api.opaali.telia.fi/sandbox/messaging/v1/outbound/tel%3A%2B358000000000/requests" --header "Content-Type:application/json" --header "Authorization: Bearer $access_token"
{
"resourceReference" : {
"resourceURL" : "https://api.opaali.telia.fi/sandbox/messaging/v1/outbound/tel%3A%2B358000000000/requests/548a1e26-326a-4c4e-a7af-8354fa6ffb24"
}
}
Sending a message to number +35812345001 simulates a successful send where you are returned a resourceURL.
POL3006 Address not in destination whitelist (tel:+35812345004)
$ curl -k -d '{"outboundMessageRequest":{"address":["tel:+35812345004"],"senderAddress":"tel:+358000000000","outboundSMSTextMessage":{"message":"Hello World"}
,"senderName":"JPLa"}}' "https://api.opaali.telia.fi/sandbox/messaging/v1/outbound/tel%3A%2B358000000000/requests" --header "Content-Type:application/json" --header "Authorization: Bearer $access_token"
{
"requestError" : {
"policyException" : {
"messageId" : "POL3006",
"text" : "The following policy error occurred: %1. Error code is %2.",
"variables" : [ "Destination Whitelist.", "3006" ]
}
}
}
POL3003 Maximum transactions per interval exceeded (tel:+35812345005)
$ curl -k -d '{"outboundMessageRequest":{"address":["tel:+35812345005"],"senderAddress":"tel:+358000000000","outboundSMSTextMessage":{"message":"Hello World"}
,"senderName":"JPLa"}}' "https://api.opaali.telia.fi/sandbox/messaging/v1/outbound/tel%3A%2B358000000000/requests" --header "Content-Type:application/json" --header "Authorization: Bearer $access_token"
{
"requestError" : {
"policyException" : {
"messageId" : "POL3003",
"text" : "The following policy error occurred: %1. Error code is %2.",
"variables" : [ "Maximum Transactions per Interval Exceeded.", "3003" ]
}
}
}
Sandbox Performance (there is none…)
One argument against doing performance testing in Sandbox Mode is that it has tighter TPS limits than Production. The benefit of this is that you’ll notice if your application behaves badly, because you’ll get the “Policy exception: POL3003 Maximum transactions per interval exceeded.” sooner than in Production Mode.
Sandbox also has a limit of 1000 transactions per day. Getting the access_token takes one transaction, but as the next time you need to do this is after the token expires after 10 minutes, this will not be a problem unless your code is broken and gets a new access_token once every (milli)second…
…and it is slow
You may have problems in Sandbox Mode if you try to send messages to multiple recipients in the same submission.
Sandbox is designed to test the functionality of the call flow. It is really only required to test with one or, if required, two numbers to check if the API request is well formed and valid, and to test responses back.
So when many numbers are added you may experience a delay in getting response from the sandbox container due to internal checks and validations. As a result of the delay experienced, on client side the application may see the error “The underlying connection was closed: An unexpected error occurred on a receive.” because the request is taking longer.
What then?
As soon as you have gained confidence in your own code you would want to move to Production Mode and continue testing in a real, live, environment.
Sandbox does protect you from your initial mistakes creating huge bills when you accidentally send hundreds of text messages, but you will only get the correct feeling how your application interacts with Opaali when you test in Production Mode.
You can test in sandbox mode which allows you to simulate different responses using predefined parameters. In sandbox mode you cannot send messages to real phones. If you want to do that then you need to promote your applications to production mode and for that you need to have a service agreement with Telia first.
JPLa is a member of the Content Gateway (CGW) to Opaali migration team, specialising in programming related issues and API usage.