When performing the solution evaluation, the business analyst assesses the solution in terms of value it brings to the organisation. The part of the work is, also, identifying potential constraints that limit the value and removing them.
Similar to the strategy analysis and requirements analysis in methods and procedure, solution evaluation differs from these two knowledge areas in the fact that the solution evaluated here actually exists. It’s not necessarily the complete solution, but still the solution that is implemented and already in use to some extent. Solutions can be evaluated in different phases of development: as prototypes, proofs, or concepts, as pilot or beta releases, or as operational releases.
The evaluation is done through performance assessments, experiments, tests and can be both objective and subjective.
Solution evaluation applies all six of the Business Analysis Core Concept Model (BACCM)
. As they evaluate solutions, business analysts recommend changes that will maximise the value of the solution and improve its performance so it can satisfy the organisational needs. The information on the solution’s performance is mainly obtained by eliciting it from stakeholders.
The BABOK lists five tasks to be performed for this knowledge area: Measure Solution Performance, Analyse Performance Measures, Assess Solution Limitations, Assess Enterprise Limitations, and Recommend Action to Increase Solution Value.
Table of Contents
Solution Evaluation – Measure Solution Performance
By measuring solution performance, the business analyst assesses the solution performance and determines how aligned it is with the overall goals of the project.
This task also includes identifying the most appropriate way to perform this assessment. When deciding on the measures to be used the BA considers the nature of the solution itself, wider organisational context, and the organisation’s definition of the desired value. The measures are often determined through collaboration with stakeholders, especially if the solution in question does not have standardised performance measures.
For input, measuring solution performance uses business objectives which represent a benchmark for solutions to be measured against, and external implemented solutions that are already in place in some form.
The outputs of this task are the established solution performance measures that can provide insight into how well the solution is performing.
The key elements of measuring solution performance are:
- Defining Solution Performance Measures
- Validating Performance Measures
- Collecting Performance Measures
The guidelines and tools that business analysts use while performing this task are change strategy, future state description, validated requirements, and solution scope. The BABOK recommended techniques for measuring solution performance are:
- Acceptance and Evaluation Criteria
- Benchmarking and Market Analysis
- Business Cases
- Data Mining
- Decision Analysis
- Focus Groups
- Metrics and Key Performance Indicators (KPIs)
- Non-Functional Requirements Analysis
- Observation
- Prototyping
- Surevery or Questionnaire
- Use Cases and Scenarios
- Vendor Assessment
The stakeholders of interest to the business analyst for performing this task are customer, domain subject matter expert, end-user, project manager. sponsor, and regulator.
Solution Evaluation – Analyse Performance Measures
Analysing performance measures involves examining the performance of a solution in order to determine the value of that solution for the organisation and to see if it satisfies the stakeholders’ needs.
This process further includes interpreting the measures established in the previous task (Measure solution Performance) to make them actionable and meaningful to the stakeholders. To perform performance measure analysis, the business analyst must first gain a full understanding of the goals and values expected by the stakeholders. Other factors to consider during the analysis are KPIs, solution risk levels, and organisation risk tolerance.
Performance measures analysis requires two key inputs: the potential value that will be achieved upon implementing the future state and solution performance measures to obtain information solution’s performance. The expected output is the solution performance analysis which also includes recommendations for solving performance gaps and maximising value through leveraging opportunities.
The key elements of performance measures analysis are:
- Solution Performance versus Desired Value
- Risks
- Trends
- Accuracy
- Performance Variances
The guidelines and tools that can be used for this task are change strategy, future state description, risk analysis results, and solution scope. The most commonly used techniques are:
- Acceptance and Evaluation Criteria
- Benchmarking and Market Analysis
- Data Mining
- Interviews
- Metrics and Key Performance Indicators (KPIs)
- Observation
- Risk Analysis and Management
- Root Cause Analysis
- Survey or Questionnaire
The primary stakeholders for performance measures analysis are domain subject expert, project manager, and sponsor.
Solution Evaluation – Assess Solutions Limitations
During the assessment of solution limitations, the business analyst examines issues related to the solutions that can negatively impact the value cause solutions not to meed the stakeholders’ needs. This assessment should discover the root causes that are the reasons for the poor performance of the solutions and solution components.
If this is the case, the business analyst examines which internal factors have caused the loss of value. There is not a set time for performing this task, it can be performed at any point of the solution life cycle no matter if the solution is still in development or completed and put to use.
The primary inputs for the assessment of solutions limitations are external implemented solutions that already exist (no matter if it’s already operational) and solution performance analysis which also includes recommendations for solving performance gaps and maximising value through leveraging opportunities. The output of solution limitations assessment is solution limitation which describes the solution limitations currently present.
The elements of this task, as listed in the BABOK, are:
- Identifying Internal Solution Component Dependencies
- Investigation Solution Problems
- Impact Assessment
The helpful guidelines and tools for the performance of this task are change strategy, risk analysis results, and solution scope. The techniques that will most likely put to use are:
- Acceptance and Evaluation Criteria
- Benchmarking and Market Analysis
- Business Rules Analysis
- Data Mining
- Decision Analysis
- Interviews
- Item Tracking
- Lessons Learned
- Risk Analysis and Management
- Root Cause Analysis
- Survey or Questionnaire
Stakeholders of importance for assessing solutions limitations are customer, domain subject expert, end-user, regulator, sponsor, and tester.
Solution Evaluation – Assess Enterprise Limitations
While the assessment of solutions limitations investigates internal factors drive the loss of value solution brings, the enterprise limitations assessment examines external factors that limit the realisation of value. The object of this task is any issue outside the scope of the solution itself that prevents it from satisfying the organisational needs.
Solutions often depend and interact with other solutions across different departments and organisational sectors and also can depend on factors that are beyond the control of the enterprise. This can include a range of external influences such as culture, technical components, operations, or stakeholders’ interests. By assessing these limitations, the business analyst identifies the root causes of these issues. Similar to assessing solutions limitation, this assessment can be performed at any moment of the solution life cycle.
The inputs for this task are current state description, external implemented (or constructed) solution that exists but is not necessarily put to use, and solution performance analysis as a result of analysing performance measures. The desired output of enterprise limitations assessment is the enterprise limitation which also describes how the performance of the solution impacts the organisation.
The enterprise limitations assessment includes the following elements:
- Enterprise Culture Assessment
- Stakeholder Impact Analysis
- Organisational Structure Changes
- Operational Assessment
The guidelines and tools usually used during this task are business objectives, change strategy, future state descriptions, risk analysis results, and solution scope. The business analysis techniques to use as recommended by BABOK are:
- Benchmarking and Market Analysis
- Brainstorming
- Data Mining
- Decision Analysis
- Document Analysis
- Interviews
- Item Tracking
- Lesson Learned
- Observation
- Organisational Modelling
- Process Analysis
- Process Modelling
- Risk Analysis and Management
- Roles and Permission Matrix
- Root Cause Analysis
- Survey or Questionnaire
- SWOT Analysis
- Workshops
The key stakeholders at this stage of the project are customer, domain subject expert, end-user, regulator, and sponsor.
Solution Evaluation – Recommend Actions to Increase Solution Value
The purpose of recommending actions to increase solution value is to identify and define actions that will increase solution value and help stakeholders better understand the differences between the potential and actual value and the undertaken actions.
During this task, the business analyst should also explore alternatives that may improve the performance of the solution. After the actions are identified, the BA recommends the steps to potentially replace, modify, or retire certain actions if that will help to maximise the value. While doing this they must take into account the long-term effect the actions may have on the organisation and stakeholders.
The necessary inputs for actions recommendation are enterprise limitation and solution limitation. They provide information on the current restraints and point out how the solution performance is impacting the entire organisation. This task, if done performed properly, should produce the output of recommended actions – the description of the steps that should be taken to increase the solution value.
The “Recommend Actions to Increase Solution Value” task consists of the following elements:
- Adjusting Solution Performance Measures
- Recommendations
The guidelines and tools that the BABOK recommends for this task are business objectives, current state description, and solution scope. The techniques usually used for the task of the recommendation of actions to increase solution value are:
- Data Mining
- Decision Analysis
- Financial Analysis
- Focus Groups
- Organisational Modelling
- Prioritisation
- Process Analysis
- Risk Analysis and Management
- Survey or Questionnaire
The stakeholders of significance for this stage of business analysis are customer, domain subject matter expert, end-user, regulator, and sponsor.
Solution Evaluation Templates
Business Analysis Doctor has supporting solution evaluation templates in a business analysis template package including a solution performance plan.
Conclusion – Solution Evaluation
Solution evaluation describes the tasks that business analysts perform to assess the performance and value delivered by a solution.
Learn more about the other IIBA business analysis knowledge areas: