Thursday, May 24, 2012

Rancho: The Code Coverage tool for ColdFusion

Code coverage is an essential part of testing, it reflects the completeness of the testing done. Code coverage is instrumental in providing information about how much of the total code is actually tested by the test suite. However code coverage does not guaranty the quality of testing but it reflects the area covered in testing.

Another important point is white box testing and code coverage(functional) are two separate activities. while the former determine the probable inputs and output for a function, the later determine if all lines and branching conditions of the function are executed, in short all paths have been covered or not. There are quiet a bunch of tools available for Java applications. But for ColdFusion applications, till now there is no way to know the code coverage.

This post provides an experimental tool, Rancho, to get code coverage for any ColdFusion application. Rancho can be used as a ColdFusion Builder extension or as an application. Download Rancho as an extension to CFB.

Rancho is the code coverage tool written in CFML for code coverage of CFML application. There are different metrics a code coverage tool can provide(line coverage,functional coverage,decision coverage) but as of now Rancho only provides the line coverage.

How to get code coverage?

There are 3 steps to get the code coverage.
  1. Instrument your CFML application using Rancho.
  2.  Run your test suite.
  3.  Generate a report of code coverage using Rancho.
Steps 2 and 3 can be iterated if you want to tun your testcases in phases.

Using Rancho as an extension to CFB

Download the Rancho and add it as an extension to CFB. View this article to add  an extension to CFB.

 Note: Since this project is hosted on Github and it provides project to download as a zip. This zip can not directly be imported to CFB as extension because extension files are one level down. So either rezip at the level of ide_config.xml OR directly download from RIAForge

You can also run Rancho as an application. Download it from here.

STEP 1 - Instrument:  Go to the Application folder in CFB navigator you need to instrument. Do a right click and select Rancho -> Instrument
Figure 1: Rancho Instrumentation
This will first take backup of the existing folder and start instrumenting all files in that folder.
Figure 2: Instrumentation Done
Note: Instrumentation is a one time activity. Do not instrument more than once.

STEP 2 - Running test suite: Once instrumentation is done you can proceed to running your testcases for the application instrumented.

STEP 3 - Getting Code Coverage Data: Once you are done with your testing or otherwise you can extract a report of code coverage. From Figure 1 click on Rancho -> Code Coverage or you can directly access http://localhost:8500/Rancho/handlers/coveragegrid.cfm
Figure 3: Code Coverage preview

This will show you the preview of your code coverage, indicating total coverage of application. Also it lists code coverage for individual files. You can further drill down to individual files to see executed and un-executed lines.

Figure 4: Code Coverage details
 This shows individual CF statements executed(green) and  un-executed CF statements(red).


Configuring Rancho

While instrumenting Rancho adds a trace before each CF statement. Now there could be some statements that can not be traced like as putting a trace before or after could be an issue. also there are some tags for which trace can not be put before e.g or .
To handle this 2 files doNottrace.txt and putTraceAfter.txt are placed at Rancho\handlers\config. Add any tag/end tag which falls in above category. As of now only some tags/end tags have been added. As you find more such tags please update them to the repository.

Limitations

  • Supports only tags not script, so any CFML code inside will be uncovered.
  • Multiple level of folders is not supported yet. Instrument and generate report for single folder level at a time.
  •  Instrumentation of single file is not supported. Only folders can be instrumented.
  • Only line coverage is provided.

Improvements

Rancho is an experimental code coverage tool so there are some improvements required like:
- Support for .
- Multi folder support
- UI improvements
- Fine tuning tags in doNottrace.txt and putTraceAfter.txt
- Enhance Rancho to extend Function and Decision coverage

I have created a github project(I am new to it). Please contribute to it.

PS:  You can also run Rancho as an application. Download it from here. Place Rancho in webroot of CF and run index.cfm. Also make sure the folder you are instrumenting has write permissions.




Saturday, May 19, 2012

ColdFusion 10: Verity to Solr migration


Now that ColdFusion 10 is out and with that support for Verity has been dropped. In CF901 both Solr and Verity were supported. But now only Solr search server is available.

If you are migrating from ColdFusion 9.0.1 to ColdFusion 10 you can convert your Verity collections in CF901 to Solr and then migrate to CF10.

If you are still on CF8 (which do not have Solr) and directly migrating to CF10(which do not have Verity), there's no direct way to migrate your verity collections created in CF8 to Solr collection in CF10.

This post provides a tool to convert your Verity collections to Solr. Download this zip from GitHub and follow instructions from readme file.

However there is a difference the way a collection is created and searched. For example, Verity requires language at collection creation time while Solr language is mentioned while indexing. See Solr in ColdFusion9 on how searching is done in Solr.