How to Select a Tuberculosis Patient Management Software Application

This guide is a set of steps for an organization that is interested in selecting a tuberculosis patient management software application. You may be initiating this process because you have identified the need for an application, because you have a tool that you want to compare to the core requirements, or because you are considering building/modifying an application. Although the steps are numbered sequentially, several of them may be executed at the same time. A glossary is provided to help users with technical terms and acronyms referenced throughout this guide.

1. Establish an Evaluation Team

When preparing to evaluate a product, it is recommended that you use a group of evaluators, preferably representing users, management, and IT. It is important to have people who understand the existing business processes (users), the future requirements (management), and the environment in which you will have to operate (Information Technolgy). If you cannot get representation from all the groups, consider appointing someone to represent that point of view. All evaluation team members should read the entire Assessment Guide as a starting point for the project. The effort for this project should be expected to take 3-6 months with weekly or biweekly meetings throughout the period. During the hands-on evaluation portion, the members may be almost entirely dedicated to the project.

Example: Begin assembling your team with representatives from users, management, and technical experts. Team members will need to have time available to lend to this effort for an extended period of time. For your user representatives, make sure all user types are represented. You may want one user who mainly does data entry, and another user whose focus is generating reports. If you are not able to have a representative from each group, one representative can gather information from others before/after team meetings. Management should be well represented on the evaluation team for future requirements and business knowledge. The evaluation team may include several IT staff. Expertise is needed for your operating environment, requirements analysis, as well as software development.

2. Understand your Organization's TB Patient Management (PM) Functions

In order to know what software to buy, you need to know what you want it to do.

The first thing to do is review the set of TB PM core use cases with your team to determine what additions or modifications need to be made to represent your particular needs for the software application. Capture these items in a list.

The TB PM Project has already defined the business processes that express the core activities of TB Patient Management. These activities are expressed as use cases and grouped into four areas: Enrollment and Registration; Care Management Cycle; Contact Tracing; and Management and Global. A use case is the specification of a function or process from the user's perspective. Each use case includes a detailed sequence of the user's actions and the system's responses. To learn more about what use cases are and how they are created, click here.

Example: Assemble your evaluation team and begin reviewing the set of TB PM core use cases. When reviewing, determine if the use case should be part of your TB PM system. For instance, Appointment Scheduling may not be one of your requirements and can be removed from your list of use cases.

3. Become familiar with the Decision Model Tool

Someone on the team should be designated the "tool jockey" - the person who maintains the group's version of the Decision Model Tool and spends enough time with the worksheets to fully understand how it operates. This individual should be familiar with Excel functions and formulas and comfortable navigating between worksheets in a workbook. Use the 1-2-3 Guide to become familiar with the various activities required to navigate, modify, and populate the tool. If you intend to modify the contents or weightings of the tool, the tool jockey will need to have the time to incorporate these changes. Be sure to allocate sufficient time to this task (at least twice that of the other team members). The Decision Model Tool will provide useful information which will become part of the purchase decision process.

Remember - you can always download a fresh version of the Tool if you encounter problems while you are learning to operate it.

Example: The Decision Model Tool can be rather intimidating at first. The evaluation team will want to designate one person with Excel experience to be the "tool jockey." The Decision Model Tool was designed to be rather flexible by letting users enter new criteria, remove unneeded criteria, and change local criteria weights. The tool includes instructions (worksheet 2) for these various tasks, but not everyone needs to know how to manipulate the tool. You will want to have one person in charge of making any changes to your version of the tool.

4. Review the list of requirements in the Decision Model Tool.

Open the Decision Model Tool to Worksheet 4 and print this worksheet. This is the "catalog" of requirements derived from the TB Patient Management Project use cases. Read each requirement to make sure you understand it. Compare your list of local needs (created in step 2) to these requirements and create or modify items to address them. This comparison will probably be completed by your entire Evaluation team. Once decisions for requirement changes have been made, changes can be made to the Worksheet 6 of the Decision Model Tool as described in the Instructions. These instructions on the mechanics of adding and changing the tool are available WITHIN worksheet 2 of the tool.

Example: The evaluation team should go through the "catalog" of requirements in the tool and compare with your list of local needs. In our example for step 2, it was decided that Appointment Scheduling is not needed. Instructions are included within the Decision Model Tool for customizing the tool. Step 2.2 of the instructions describes how to remove (gray out) criteria. These criteria will still be visible in your worksheet, but they will appear gray and will not be included in worksheet calculations. Appointment Scheduling criteria are 4.5.1 through 4.5.5 in the Decision Model worksheet (worksheet 6 of the tool). Select the rows for 4.5.1 through 4.5.5, choose format cells, and change the text color to gray. The next step is to change the local weight drop-down list to "NA" for each of these criteria 4.5.1 through 4.5.5. This criterion is now effectively removed from your decision model.

5. Review the "local weights" of the requirements in the Decision Model Tool.

Now that you have a "complete" list of requirements, review the local weights assigned to make sure they match your local priorities. Because there are many requirements, each individual one does not carry much weight, so you may find that you are comfortable with the weightings as provided. You will have to adjust the weights in areas where you have added (or removed) requirements.

Example: Each requirement/criteria in the Decision Model Tool has been assigned a local weight. You may find that some of these local weights do not meet your needs. Section 2.3 of the Decision Model Tool Instructions contains information on Modifying Weights. For instance, section 4.9 addresses requirements for Care Plan Management. The weight of requirement 4.9.1, "Define, Record, and/or Construct a Care Plan" is currently designated as "Should Have." Your evaluation team has decided that the Care Plan is crucial to your new application and this requirement will be changed to the highest weight of "Must Have". This is done by selecting the "Must Have" value in the drop-down list for this requirement. No other changes are necessary.

6. Select software packages to evaluate.

You may already have identified software application packages that you are interested in evaluating. If not, start by reading the sections on Business Models and Support to help you understand the options available to you. You may want to use the results of the TB PM Project or the Turning Point site as a resource to identify products to review. We suggest you limit your initial investigation to 3 - 5 applications and only include additional products if the initial applications fail to meet your requirements.

Example: As an Evaluation Team, application packages should be discussed and a consensus reached about which applications to evaluate. You may already have a current system you are using which you want to evaluate to determine needed enhancements.

7. Request information

Contact each software provider identified in Step 6 to request information on the functionality and pricing of their system. Besides brochures and documentation, you should also request a demonstration of their system that allows you some hands-on access to the application to assess functionality. Pricing information may require you to provide information on users, installation approaches, or other factors. At this point, you are mostly interested in understanding the components of the price. (see Pricing Models)

Example: You will want to make a request in writing to the software application provider requesting information about their system. The questions from worksheet 4 of the Decision Model Tool can be copied into a document to be sent to potential software application provider to get their feedback about their capabilities. You may also want to mention to the software application provider that you will want a demonstration in the future which includes some hands-on access to the application.

8. Initial evaluation

Prior to the demonstration, read through the documentation provided and score as many items as possible. You may want to do the evaluation with your entire Evaluation team together so that consistent rules are used when answering all questions. You should answer questions when you have "verifiable evidence" and note what that evidence is in the comments column (i.e. citation of a page of a user manual, validation from an installed user site, reference to a screen in the application). If you have the "answer" to a requirement, but no evidence to back it up, set the credibility value to "low". From this step you will end up with a list of requirements that you cannot evaluate from the descriptive material and also a list of questions that you want answered. Provide these lists to the software application provider prior to the demonstration.

Example: Each member of the team should answer questions independently before the evaluation team gathers to perform the initial evaluation. The evaluation team should gather and come to a consensus when answering all questions and putting data into the Decision Model Tool.

9. Schedule a hands-on demonstration of each product.

Use this demonstration session to gain answers to your questions and verify the presence or absence of functionality. Evaluate each application on the same basis - past, current, or future functionality. Since the evaluation represents a "point in time", it is important that each application be represented in an equivalent manner. Make sure this demonstration allows you to have hands-on experience. There is a big difference between watching someone else use an application and in sitting at the keyboard yourself. You may want to request access to the application for a period of time to allow more extensive testing. Testing should consist of a pre-determined set of steps that you will attempt to execute. These steps and the data to support them are called a Īscenario.Ķ Each system should be tested using the same scenario(s).

If you choose to consider an application that is not yet available, based on written descriptions or requirements, note that in the credibility rankings and the comments section.

Example: This is where you can try out specific actions that your jurisdiction considers key or perhaps locally-significant. Enter the problem case or the typical situation so you will have a feel for the actual use of the product.

10. Consider the external factors.

Although the Decision Model Tool includes categories for a number of factors, its results should not be the sole basis for a purchase decision. The team should list those factors that will make an implementation successful. Start by considering the many other items discussed in this Guide (i.e., Data Migration, Pricing Models).

Example: Acquisition or funding restraints may lead you to choose an application that was developed by another governmental jurisdiction. The ability to customize the field names to match local usage or language may be critical. These items should be listed and considered.

11. Check references and get prices.

When you've narrowed it down to 1 (or maybe 2) applications, check their references. Don't talk to just one person at a site, get a counterpart for each of the components of your team (users, management, IT) and gather their impressions. This information will be extremely helpful since these references are using the application in their environment instead of running a demonstration version. At this point, you will need to provide the supplier the required information to get a price quote. Depending on the requirements of your organization, this may be done by your purchasing organization rather than the evaluation team.

Example: Contact references and set up appointments for discussions (internal or external). This information will allow your organization to get a "real world" perspective of the software application. These references can give you information regarding response times, ease-of-use, reporting capabilities, and product release schedules.

12. Make a recommendation

You're done! Present your findings to your purchasing authority. You may want to attach the ratings entered into the tool as part of the purchasing justification.

Example: You may decide that your current systems meets most of your needs, but needs a few enhancements. Or, an application exists which meets all of your basic needs and customizations can be done for future requirements.

Skip Navigation | Assessment Guide | How to Get Started | Requirements Definition | Pricing Models | Support | Data Migration
Local Enhancements and Customizations | Costs | Funding | PHIN Compatibility | Decision Model Tool | Glossary | Results

Establish an Evaluation Team Understand your Organizations TB Patient Management (PM) Functions Become familiar with the Decision Model Tool Review the list of requirements in the Decision Model Tool Review the local weights of the requirements in the Decision Model Tool Select software packages to evaluate Request Information Initial evaluation Schedule a hands-on demonstration of each product Consider the external factors Check references and get prices Make a recommendation