CAT on the Brink of 1 Million Recorded Coding Choices

Texifter manages the Coding Analysis Toolkit (CAT), which is a free, open source, Web-based and FISMA-compliant system launched in the fall of 2007 and hosted by the University of Pittsburgh. CAT is the precursor to PCAT and DiscoverText.

This is a big day for the CAT team, as we are on the brink of recording the 1 millionth coding choice in the system.

Why do people like to use this software? Certainly the price helps. However, over the years we have engineered CAT to make some of the most common coding and validation tasks easier. CAT uses a simple keystroke coding interface and predefined text spans to limit the pain caused by using a mouse.

More important to regular CAT users are the on-board tools for easily calculating multi-coder reliability. CAT simplifies the process of assigning the same coding task to a group of coders who can code asynchronously via the Web.  When the coding is done, it is a simple matter to generate table of rater reliability stats for better understanding how different coders use the various codes. When pre-testing a new coding scheme, this on-the-fly measure of reliability is a key learning and training tool we use in QDAP all the time.

Probably the most important innovation introduced by CAT is the adjudication module. The 118,850 adjudication choices recorded to date by CAT users grew out of our practice of comparing multi-coder experiments pen and paper. Aside from using lots of paper in big experiments, we found ourselves with a time-consuming challenge to transform our validation choices back into the software we were using at the time.

Validation in CAT allows an expert or consensus team to review coding choices one at a time and score them as valid or invalid. The system reports validity as a percentage by code, coder and project. This (often iterative) step is absolutely critical when training a coding team.

About Stuart Shulman

Stuart Shulman is a political science professor, software inventor, entrepreneur, and garlic growing enthusiast who coaches U13 boys club soccer and in the Olympic Development Program with a national D-license. He is Founder & CEO of Texifter, LLC, Director of QDAP-UMass, and Editor Emeritus of the Journal of Information Technology & Politics. Stu is the proud owner of a Bernese/Shepherd named "Colbert" who is much better known as 'Bert. You can follow his exploits @stuartwshulman.
This entry was posted in general and tagged , , , , , , , . Bookmark the permalink.

Comments are closed.