JSON Tools - jq

Comparing with Python json.tool, seems jq is much light weight and powerful :)

Sample time diff:

With jq:
real 0m1.666s
user 0m0.262s
sys 0m0.290s 

With Python json.tool:
real 0m2.217s
user 0m0.492s
sys 0m0.539s


Useful links:

• Comparing Command-Line JSON Pretty Printers
• jq is sed for JSON
• jq play

HTML parsing with Python

Scraping Data with Python and XPath:

Sample code from reference link which tell the whole story :)

import requests
from lxml import html

pageContent=requests.get('https://en.wikipedia.org/wiki/List_of_Olympic_medalists_in_judo')
tree = html.fromstring(pageContent.content)

goldWinners=tree.xpath('//*[@id="mw-content-text"]/table/tr/td[2]/a[1]/text()')
silverWinners=tree.xpath('//*[@id="mw-content-text"]/table/tr/td[3]/a[1]/text()')
#bronzeWinner we need rows where there's no rowspan - note XPath
bronzeWinners=tree.xpath('//*[@id="mw-content-text"]/table/tr/td[not(@rowspan=2)]/a[1]/text()')
medalWinners=goldWinners+silverWinners+bronzeWinners

medalTotals={}
for name in medalWinners:
    if medalTotals.has_key(name):
        medalTotals[name]=medalTotals[name]+1
    else:
        medalTotals[name]=1

for result in sorted(
        medalTotals.items(), key=lambda x:x[1],reverse=True):
        print '%s:%s' % result

BeautifulSoup is another option but different style from xpath.

HTML page parsing with xmllint xpath in BASH

Per HTML_parsers, there is no better HTML page parsing options for BASH. Inspired by Retrieve web using xpath, here comes the summary of using xmllint xpath:


xpath='' # sample: '//div[@class = "tides"]'

get_element_by_xpath():
    echo $HTML_PAGE | xmllint --html --xpath $xpath - 2>/dev/null

get_element_text_by_xpath():
    xpath+='/text()'
    echo $HTML_PAGE | xmllint --html --xpath $xpath - 2>/dev/null

get_elements_count_by_xpath():
    xpath="count($xpath)"
    echo $HTML_PAGE | xmllint --html --xpath $xpath - 2>/dev/null

Setup Bash Debugger in VS Code

Recent projects need some bash scripting, besides bashdb in terminal, GUI bash debugging make life much easier:

1. Follow all steps in Upgrading Bash on macOS to update bash (need bash > 4.0)
2. Install bashdb: brew install bashdb
3. In VS Code:
   1. Install extension Bash Debug
   2. Configure bash debugger by https://marketplace.visualstudio.com/items?itemName=rogalmic.bash-debug
   3. Additional configure: add "terminalKind": "debugConsole" to launch.json to each configuration

API Automation Test

As per Top 10 API Testing Tools, API Test becomes a buzz word now because many companies nowadays require strong API performance and shifts towards APIs architecture, hence this summary post of API test.

Use Katalon

As I said in earlier post, I'm a fan of Katalon Studio which combines UI and API test in one Eclipse based user friendly IDE with valuable Recorder(record/play) feature! Only missing part is no direct support of performance test.

As shown in Create your first API test with Katalon Studio (manual style without scripting) and Katalon Web Service Test (video with a little bit scripting), Katalon supports API test in powerful user friendly way as other popular API test tools e.g. Postman, Restlet Client; By Parameterize a Web Service object and scripting, we could have all kind of APIs calls combinations.

The important point here is Katalon uses Groovy as scripting language, besides enjoy the beauty of Groovy, we have full control of the scripts flow! We could introduce any other external Java libraries into Katalon Studio to leverage all powerful features on Data-driven, assertion etc. integrating with UI/API testing. Even make up some performance test indirectly with coding around 3rd party libraries e.g. REST Assured.

Use long live JMeter

And still love JMeter if load test API after feature test is a must. JMeter is born for load test!

• JMeter Performance and Load Testing @ DZone (the detailed basics of JMeter)
• Parameterize sample tutorial 1 | tutorial 2
• Advanced data driven: JMeter – REST API Testing – A Complete Data-Driven Approach

Pure Coding style using REST Assured

With REST Assured along come out home brew BDD style API test! Recommend tutorial.

Geb / Spock Gatling vs JMeter

Geb

is a browser automation solution. GEB = Selenium WebDriver + Java + Groovy + Jquery

It brings together the power of WebDriver, the elegance of jQuery content selection, the robustness of Page Object modelling and the expressiveness of the Groovy language.

It can be used for scripting, scraping and general automation — or equally as a functional/web/acceptance testing solution via integration with testing frameworks such as Spock, JUnit & TestNG.

Spock (same idea as Cucumber): 

Spock is a testing and specification framework for Java and Groovy applications. 

Spock lets you write specifications that describe expected features (properties, aspects) exhibited by a system of interest.

Gatling:  (vs Jmeter )

Gatling is a highly capable load testing tool. It is designed for ease of use, maintainability and high performance.

Out of the box, Gatling comes with excellent support of the HTTP protocol that makes it a tool of choice for load testing any HTTP server. As the core engine is actually protocol agnostic, it is perfectly possible to implement support for other protocols.

Gatling simulation scripts are written in Scala.

JMETER VS GATLING TOOL: https://octoperf.com/blog/2015/06/08/jmeter-vs-gatling/

Selenium WebDriver Automation Best Practices (Refer to testautomationguru)

Notes while learning from testautomationguru.com (THANKS!) which need strong development skills ;) With help of Katalon, seems most best practices could be implemented :) Keep the summary to remind best practices will always help!

1. Page Objects Model, + Arquillian Framework

1. Page Objects: Page objects is a well known design pattern, widely accepted by the automation engineers, to create separate class file for each page of the application to group all the elements as properties and their behaviors / business functionalities as methods of the class. But it might not be a great idea always, especially when the page has more / different sets of elements / complex element like a grid / calendar widget / a HTML table etc. ex http://www.testautomationguru.com/keyword-driven-framework-for-localization-testing-using-selenium-webdriver/ (Table/keyword driven + page objects model)
2. Selenium WebDriver – Blackbox Automated Testing using Arquillian Framework 
3. Selenium WebDriver – Advanced Page Object Pattern with Page Fragments using Arquillian Graphene (Single Responsibility Principle)

2. No hard code of test data: Selenium WebDriver – How To Design Tests & Test Data

3. Design patterns in Test Automation: http://www.testautomationguru.com/category/design-pattern/

"Note: Your aim should not be to implement a certain pattern in your framework. Instead, identify a problem in the framework and then recognize the pattern which could be used to solve the problem. Do not use any design pattern where it is not really required!!"

• Creational patterns - Factory Pattern (Driver Manager Factory)
• Behavioral patterns - Strategy Design Pattern (Text or Voice search)
• Behavioral patterns - Template Method Pattern (Car and Hotel reservation)
• Structural Patterns - Proxy Pattern (Run selected test on Production env.)
• Structural Patterns - Decorator Design Pattern (Decorator pattern allows us to add additional responsibilities to an object dynamically. By using decorator pattern, we create page objects with dynamic components based different roles. In future, if the application adds another component or another role type, it is extremely easy to update the test.) (Used Java8 Consumer Functional Interface for better readability)
• Behavioral Patterns - Chain Of Responsibility Design Pattern (Whole online shopping process)
• Execute Around Method Pattern (execute around method pattern hides the complex logic of switching to frames / windows etc. The user does not have to worry about switching and instead just focuses on business logic. It makes both page objects and test classes look clean and neat.)

Refer to https://sourcemaking.com/design_patterns or http://www.oodesign.com/ for design patterns.

4. WebDriver in Fluent Style (java 8 Stream)

Selenium WebDriver – How To Design Page Objects In Fluent Style

Selenium WebDriver – How To Design Business Workflows In Fluent Style

5. Test Suite Management:

Selenium WebDriver – How To Automatically Skip Tests Based On Test Environment

(By creating a custom annotation and using TestNG’s listener, We are able to skip methods without any issues and maintains the order the test execution.)

Or by Selenium WebDriver – Design Patterns in Test Automation – Proxy Pattern (Filter test env in proxy)

Katalon Introduction

Katalon

Recently find free (not open source) automation tool Katalon and love it, personally believe it will be more popular in near future, recommend everyone interested in automation no matter beginner or expert to take a look!  (No pay from Katalon for voluntary :D )

See what Wikipedia says about Katalon:

• "The software is built on top of open-source automation frameworks Selenium, Appium with a specialized IDE interface (implemented on Eclipse) for web, mobile and API testing." 
• "The main scripting language is Groovy, Java, and JavaScript and can be executed against all modern browsers, iOS, and Android applications supported by Selenium and Appium." 
• "Katalon Studio follows the Page Object Model pattern." 
• "The remote execution can be triggered by CI systems via Docker container or command line interface (CLI)."

Cool, right?! One more very important point missed on Wikipedia is the record and play feature. It's normally sale point of commercial automation tool (HP UFT/QTP, IBM RFT, TestComplete etc.). As a starting point or code snippet, it is a great time saving feature.

Object recognition support multiple methods to get job done which make scripts more tolerate to AUT changes, no as good as IBM RFT's but good enough.

For Selenium 1.0 IDE lover, Katalon Automation Recorder Chrome extension is the latest replacement.

Well organised Official Katalon site has all necessary information (docs, forum etc.)

There is a free Udemy course Automate everything with Katalon Studio.

IBM RFT: how to invoke object methods

Looks like currently IBM RFT only support Java domain objects methods invoke(?)

Three steps to invoke the object methods:

1. get all methods signature:

printTOMethods(TestObject TO) {

int i = 0;

System.out.println("Object Class [" + TO.getObjectClassName() + "]'s methods: \n");

for (MethodInfo mi : TO.getMethods()) {

System.out.println("Method-" + (i++) + ": [" + mi.getName() + ":" + mi.getSignature() + "]");

}

}

2. Study api documents to select the methods you need to use according to your target;

3. invoke object methods strictly obey the signature:

for example, for some 'new' objects RFT could only blindly click/select by screen point, you need to find the screen point for RFT:

boolean selectTableItem(ScrollTestObject table, String itemName) {

boolean selected = false;

TestObject[] itemsTo = (TestObject[])table.invoke("getItems");

for (int i=0; i

if (itemsTo[i].getProperty("text").toString().equalsIgnoreCase(itemName)) {

java.awt.Rectangle rec = (java.awt.Rectangle) itemsTo[i].invoke("getBounds", "(I)", new Object[] {new Integer(0)});

table.click(atPoint(rec.x + rec.width/2, rec.y + rec.height/2)); //click at the item center to select it

selected = true;

break;

}

}

return selected;

}

IBM RFT: Enable Java Domain for Java AUT

Normally IBM RFT just regards all objects in Win domain without respect that the AUT is a Java application.
It's fine normally, but for some special objects (e.g. grid) against which you'd like call the special methods for your test purpose, you could enable RFT to treat objects in Java domain:
In RFT, 'Configure->Enable Environments for Testing...';
In tab 'Eclipse Platforms', 'Add...' browse to where your Java application home_dir (with private jre);
'Enable' it, then 'Apply', 'Finish' to close 'Enable Environments' window.

JMeter HTTPClient HTTP sample might be the reason of jmeter crash

As statement in some document that HTTPClient HTTP sample is more stable than simple HTTP sample, but it might also be the reason why jmeter crash since HTTPClient HTTP sample log too many messages in jmeter.log like below,

2007/08/20 16:05:35 INFO - org.apache.commons.httpclient.HttpMethodDirector: Redirect requested but followRedirects is disabled

In this case have to change back to simple HTTP sample:(

MS-DOS Batch Script Tips

Never suppose to write windows bat file. But when recently try it, even it really ugly comparing with unix shell scripts, still something interesting to learn and enjoy:)

rem setlocal not to impact environment variables
setlocal

rem get current disk driver and directory in case needed resources in specific location
set workDriver=%~d0
set workPath=%~p0
%workDriver%
cd %workPath%

rem call some command/executed program here
rem then check ERRORLEVEL for error handling
if ERRORLEVEL 1 (set errorCode=1
rem change color for warning and fun :)
color 4e
echo Error encounter!
pause
color 0f
goto error)

Useful link:
Batch File Command Reference for Windows 2000
MS-DOS/MSDOS Batch Files: Batch File Tutorial and Reference

Jmeter: Summary Report vs Aggregate Report

When using Aggregate Report listener in Jmeter script, the memory usage will increase linearly with the number of total samples grow. Which might be the biggest reason JMeter got out of memory for long period run. Reason is,

Warning in Aggregate Report help: "Calculation of the Median and 90% Line values requires a lot of memory as details of every Sample have to be saved. See the Summary Report for a similar Listener that does not need so much memory."

So use Aggregate Report only when you care Median and 90% Line values. But what Median, do you really care it, good explanation is here.

JMeter: BeanShell Example

For example, I need to handle number's get from web page which is return by HTTP Request Sampler. It's hard to get the number string directly from ResponseData sometime, just use a "Regular Expression Extractor" and store it to a temp variable, say _tmpAmount, then,

Script:

String amountString = vars.get("_tmpAmount");
int amount = Integer.valueOf(amountString).intValue();

int minAmount = 10;

if (amount > minAmount) {
vars.put("_finalAmount", Integer.toString(minAmount));
} else {
Failure = true;
ErrorMessage = "Must has at least " + minAmount + " in original account!";
SampleResult.setDataType(SampleResult.TEXT);
SampleResult.setResponseMessage(ErrorMessage);
SampleResult.setResponseData(ErrorMessage.getBytes());
SampleResult.setStopThread(true);
}

JMeter: Dynamic SOAP XML data of WebService(SOAP) Request

SOAP XML data of WebService(SOAP) Request could be set by,

a). "Soap/XML-RPC Data" text field directly or
b). reading from file or
c). randomly picking up from directory when data files stored.

When want to send dynamic soap xml data, say different user name, method a) doesn't work since it's hard coded; method c) works here as long as we generate the data files in advance.

How about method b)? It looks more neat, at least for me:) But how?

The way I could think about is using a Java Request before each WebService(SOAP) Request. In the Java Request, just use simple BufferedWriter writer = new BufferedWriter(new FileWriter("soapDataFile")) to generate the soap xml file which includes the dynamic user name variable read from user file. Since the xml is not complex, just regard it as text file, no bother to use the more professional heavy XML processing method like How to Read and Write RSS Feeds (Linked here just in case)

JMeter: BeanShell Usage

With JMeter, sometimes we need run to later samplers based on response from previous samplers to provide scripts' flexibilities.

Normally "Regular Expression Extractor" could get expected value from sampler response;

Otherwise we might need to use __BeanShell(), __javaScript() (examples) with the help of JMeter's "Options | Function Help Dialog";

For more complex cases, we might have to use "BeanShell Assertion"; How to debug it? Refer to BeanShell Manual and try it in BeanShell Desktop window before copy it to your "BeanShell Assertion". Where to start BeanShell Desktop? Download it from http://www.beanshell.org/download.html and save it to your jmeter_home/lib, just double-click bsh*.jar and play. Enjoy it:)

JMeter: HTTP Proxy Server doesn't support HTTPSampler2

In current JMeter version 2.2, in jmeter.properties's "JMeter Proxy recorder configuration" section, "Change the default HTTP Sampler" by removing "#" from line jmeter.httpsampler=HTTPSampler2, the HTTP Proxy Server supposed to recording the HTTP sampler by using HTTPSampler2, i.e. HTTPClient HttpSampler.

But unfortunately, we will got "ERROR - jmeter.protocol.http.proxy.Proxy: java.lang.NullPointerException" in jmeter.log and gateway timeout in browser. By checking http://article.gmane.org/gmane.comp.jakarta.jmeter.user/14834/match=jmeter+protocol+http+proxy+java+lang+nullpointerexception+httpsampler2 we could know, it's still not supported:(

OCR in TestComplete

OCR which might be the last choice for identify object/text in screen in TestComplete.

Open OCR sample at,

C:\Program Files\Automated QA\TestComplete 5\Samples\Scripts\OCR\OCR.pjs

Just run it.

There are 2 parts of code, but below is the key point,

TextToFind = "E-mail";

Pic = wOfficeListBox.Picture(0, 0, wOfficeListBox.Width, wOfficeListBox.Height, false);

OCRObjList = OCR.CreateObject(Pic);

FindRes = OCRObjList.FindRectByText(TextToFind);

The only part you need to replace is Pic = wOfficeListBox.Picture(0, 0, wOfficeListBox.Width, wOfficeListBox.Height, false);

You could use the way in this sample if you could get the object, otherwise you could always use Sys.Desktop.ActiveWindow() to get the window snapshot and specify the coordinates of the area you care, and do the OCR, the recognition speed is good.

And check TestComplete Help for topic “Recognition Tips” to know which area is better for your case.

TestComplete Notes: big page handling & memory saving tips

1. Method to identify big page

var childNum = doc.ChildCount;

if (childNum > 13000 || childNum == 0) {

Log.Warning("Can not handle too large page!");

return null;

}

2. Release no use memory for browser and TestComplete

Based on information at,

http://www.automatedqa.com/forums/d.cgi?cmd=article&group=automatedqa.public.testcomplete&item=20066&utag=

minimize then maximize window could force process to release the no used memory, code for IE is as below:

// to release memory of browser

var browerWindow = windowPage.Parent;

while (! BuiltIn.IsSupported(browerWindow, "Minimize")) {

browerWindow = browerWindow.Parent;

}

browerWindow.Minimize();

browerWindow.Restore();

// to release memory of TC

Sys.Process("TestComplete").Window("TfrmTCMainForm", "*", 1).Activate();

Sys.Process("TestComplete").Window("TfrmTCMainForm", "*", 1).Minimize();

Sys.Process("TestComplete").Window("TfrmTCMainForm", "*", 1).Maximize();

you could do above at your threshold by checking Sys.Process("TestComplete").MemUsage

But for FireFox, to release the memory, we are not lucky. We have to use task manager to minimize and maximize it. In addition, we also have to turn on MSAA and hard code like below,

function _releaseFFMemory()

{

var p1;

var w1;

var w2;

p1 = Sys.Process("Explorer");

p1.Window("Shell_TrayWnd").Window("Button", "start").btn_Start.Click();

p1.Window("DV2ControlHost", "Start Menu").Window("DesktopSFTBarHost", "", 2).Window("SysListView32").mi_Run.Click();

p1.Window("#32770", "Run").Window("ComboBox").Window("Edit").Keys("taskmgr[Enter]");

p1 = Sys.Process("taskmgr");

w1 = p1.Window("#32770", "Windows Task Manager");

w2 = w1.Window("SysTabControl32", "Tab1");

w2.ClickTab("Applications");

w2 = w1.Window("#32770", "", 1).Window("SysListView32", "Tasks").list_item_Bookie_Home_Page_Mozilla_Firefox;

w2.ClickR();

p1.Window("#32768").mi_Minimize.Click();

w2.ClickR();

p1.Window("#32768").mi_Maximize.Click();

w1.title_bar.btn_Close.Click();

}

3. Use as little memory as possible

Tips from the same message board and my experiences,

1) Log message only when needed;

2) Turn on auto-save log: Options | Engines | Log and set the auto-save interval. This will flush the log to disk every x number of minutes. (better performance, not sure if help reduce memory usage)

3) Store screenshots in PNG format instead of BMP format.

4) Call the Log.LockEvents method to prevent TestComplete from posting event messages; And call Log.UnlockEvents when you need it. If you never care event messages even when you got warning or error, use Log.LockEvents(0).

JMeter: Recommend using HTTP Request HTTPClient

Use HTTP Request HTTPClient instead of the normal HTTP Request Sampler in JMeter for load test. There is a known limitation for the latter because it uses the default java.net and java.io connections which do not scale well in long running tests. The HTTPclient sampler on the other hand uses the Jakarta HttpClient library which is more efficient and stable.

By checking HTTPSampler2 source code which is for HTTPClient HttpSampler in package org.apache.jmeter.protocol.http.sampler, we could find more checks/controls than normal HttpSampler in same package. Even we don’t care the detailsJ

Conclusion: use HTTPClient HttpSampler but not normal one. They have the exact same GUI.