Hi everyone. I would like to introduce TerraSky’s method of testing we do for the SkyVisualEditor platform — a semi-automated user permission testing. I will also go over the tools we use to achieve this.
Since SkyVisualEditor is a tool to generate Visualforce pages, we must run tests to check if the pages do follow the user profile and permission settings that are configured on the Salesforce organization.
We usually run about 200 tests on different Visualforce pages for each user profile.
The Automated Testing
Below is a list of tasks we do for the test:
Salesforce configuration
Create/Deploy metadata of Test Object
Create/Deploy metadata of Page Layout
Create/Deploy metadata of Object and item’s permission
Grant permission to the test user profile to access every Visualforce page in the Salesforce organization
Test data
Create
Delete
Create SkyVisualEditor page data
Duplicate a template page data to every object
Test the deployed page by using different users, create new page and edit page. Save screenshots for each tested pages.
Granting permission to access a Visualforce page can be a hassle because deleting a beta version package will require deleting all of the pages and then doing a retest. This will stop the deploy process until all of the pages with the original profile metadata are deployed. Hence, to make this easier, you can open a profile setting page from an automated browser and select all of the pages that have been deployed to test the selected pages.
The Manual Testing
The automated testing covers tasks up to screenshot takings. After that, we go through manual checking. We do not automate assertion testing because of the time and possibility of changing the page.
Tools
We use the following tools to create tasks for the tests.
JSforce
JSforce is a client tool that can access the Salesforce API from node or browser. It is compatible with many APIs. We use this tool to create and delete test data, deploy metadata for salesforce organization, and to retrieve metadata for different management. Find more details here.
Grunt / gulp
Grunt and gulp are task runners that work on a node. They pick up tasks from npm or run tasks that are set for the project. We only use zip/unzip plugin and use JSforce or Nightmare to run the tasks. However, it is nice to have – to attach help reference (README file) to each task.
Originally, we used Grunt + grunt-ant-sfdc to deploy. However, it did not run on Windows OS. So based on the following information below, we switched to gulp + JSforce. (While writing this blog, I realized that it is a plugin for gulp-jsforce-deploy)
At first, we were worried about its API consumption when creating and deleting test data. So we used Ajax Toolkit in CasperJS. Now, we use JSforce to create gulp tasks. (We use Apex code with executeAnonymous to delete all records to reduce the API consumption.)
PhantomJS
PhantomJS is a headless WebKit scriptable with a JavaScript API. You can find similar tools like SlimerJS and trifleJS.
We use the wrapper tool below because there is too much fuss to go over every page using PhantomJS API.
CasperJS / Nightmare
CasperJS and Nightmare are PhantomJS wrappers. There is also a Horseman.js which I have not tried out.
CasperJS and Nightmare are PhantomJS wrappers that use JavaScript for navigation scripting but the code is conducted in a different environment.
CasperJS is distributed as an independent command, not as a node module. So, there are restrictions such as npm modules that cannot be used or the scope is the same as the designated requirement. This is because the code for CasperJS uses a similar context as PhantomJS.
On the other hand, Nightmare is an npm module which can be used with node requirement. Therefore, various npm assets can be used (CasperJS can be used with nodes that have SpookyJS). However, there are problems with Nightmare because the evaluation process uses a callback system, so you might have to deal with a real callback-nightmare if you want to reuse the value retrieved by scraping to another page.
Although I use CasperJS I am planning to use Nightmare if the number of test patterns increased or if there is a need to automate assertion.
Summary
As a conclusion, I suggest using handy tools that are out there to automate and reduce the burden of testing. It will help you from simple human-errors like forgetting to do one of the test tasks.
Image courtesy of tiramisustudio at FreeDigitalPhotos.net
Comments