Answering Questions: Getting the Right Data to the Right User

Application Portfolio Management helps IT professionals make better decisions about the systems that run their business. But who are these decision-makers and what kinds of choices are they actually making? This posting will take a look at that topic.

Decision-making is not the exclusive purview of senior leadership. Every day architects, analysts, development managers, and developers are making fundamental choices that affect the service-levels provided by applications:

• Architects want to determine how well architectural models have been implemented within applications. Is there a high-degree of dependency between architectural entities? Where is complexity eroding the flexibility of my systems?
  
• Analysts need to understand how well their business processes are maintained by development. Are changes executed quickly with limited rework? How costly are these changes, and how costly overall are the applications that run their lines of business?
• Development and outsourcing managers want to understand which teams are most effective and which aren’t pulling their weight. Where is complexity rising in the portfolio, and where should refactoring be launched?
• Operations managers need to make the linkage between systems that fail or are non-performant and the applications and data stores that run on them. Where can improvements be made? Where are duplicate and redundant systems that can be turned off?
• CIOs want to know overall costs associated with applications, their development, and their underlying infrastructures. In fact, they likely want to see all of the above information, but at an appropriately high level of abstraction.

It is clear that as we go through the goal / question / metric paradigm that there will be different goals for different user roles and levels in the organization. So, it is only natural that the type, focus, and summary level of this data will vary depending on the user and their goals and questions.

Now the question becomes ‘how do we get the data that each of the consumers needs?’. In the previous posting, I looked at what data sources are useful. User surveys, external tools (like a PPM or ALM toolset), and analysis of source code are the key data sources. Also as previously mentioned, they should be included and weighted in varying degrees based on the questions being asked.

Filter Answers at the Right Level
But how do you filter your results for each role and level? The key is the concept of abstractions. Essentially this practice involves defining models that align with how users think about their organization. For business analysts that could be by overarching business process and then by sub-process. For development managers it could be by development team and scrum team, or by outsourcer.

This process doesn’t have to be exhaustive or complex. It just has to define groupings of IT assets that make sense to users. In some cases this is already done through activities like ITIL. In other cases, simple discussion with stakeholders is sufficient to provide the right level of detail, as you can see in the model below.Why do we need these abstractions? Because they provide the “buckets” into which data is sorted as it arrives. These buckets will frequently intersect, so a “claims processing” system that is interesting to an analyst could be managed by “Outsourcer A and B”, which is interesting to a development manager. A simple matrix like this allows data to be quickly sorted to be relevant to different users.

Then, as data arrives from one of your three data sources, it can be marked as being relevant for different types of users. So, complexity data about an application can be marked, preferably in an automated fashion, as being relevant for a development manager and for an architect, say.

Presenting Data Back to Users
As you collect, group, and store data, users will want to access it to support decision-making. Your reporting mechanism, whether that is a purpose-built reporting tool or not, should use the groupings that you have defined. That means reports will filter based on the groupings that are relevant to the end-user and geared to answer their specific questions.

For instance, the development manager that wants to determine which teams are performing. He might combine application complexity measures with bug count data, filtered by scrum teams X and Y. His boss might pull the same data, but instead across the broader data set for Outsourcer A and B. Presenting the right level of data to the right user to help answer their questions and support their defined goals.