Panaya

SAP ERP Lifecycle Management: Overcoming the Downside of SAP Upgrades
By Dan Yachin
Download the IDC White Paper in PDF Format

Panaya's introduction:  SAP upgrades bring significant challenges pertaining to managing project expenses and controlling project scope. While SAP users see technical and functional benefits from upgrading to ERP 6.0, there is a downside. Upgrades can be time consuming; identifying the potential problems with interfaces and testing systems appropriately can be an exhaustive task. In this white paper, IDC defines the key challenges SAP users face during the SAP ERP 6.0 upgrade process. IDC then assesses the viability of Panaya's automated SAP upgrade solution to alleviate these problems and streamline the SAP upgrade process. 

IDC OPINION
The ongoing economic crisis is causing many organizations to rethink strategic initiatives and projects related to critical business applications. Driven by the need to improve cost control and predictability, enterprises are reluctant to take on projects that require lengthy implementation time.

In the ERP market, this sentiment has a direct impact on the intensions of many SAP users that are currently considering upgrading to the latest SAP ERP 6.0 version. While most organizations continue their upgrade projects as planned, mainly due to reasons such as end of maintenance and functional requirements, reducing the project costs becomes acutely important. However, given the significant complexity of ERP SAP systems, completing an upgrade project in a timely and cost-effective manner is highly challenging.

Quite ironically, one of the main reasons for the complexity of SAP ERP is its flexibility in allowing users to fine-tune and customize the system to address their particular needs and procedures. This flexibility trades-off with ever-increasing code complexity, which in turn results in inability to accurately predict whether or not the multiple customizations and changes will operate properly when upgrading to the new version or implementing other major changes.

In order to avoid potential errors that may affect key application functionalities and disrupt critical business processes, SAP ERP users are required to perform comprehensive tests before going live. Traditionally, organizations have been dealing with this challenge by using a manual trial-and-error approach to test their customizations. However, the shortcomings of this method are exacerbated the more complicated the ERP system becomes. This includes, for example, inability to detect all problems, mounting costs due to increased utilization of development and testing resources, extended implementation time, and inability to accurately calculate the project timeline and costs.

To tackle the shortcomings of traditional approaches, there is a growing need for solutions to automate testing processes. These solutions should provide comprehensive capabilities such as impact analysis of customizations and changes, as well as automatic detection of potential problems across the entire application life cycle – from development to configuration changes, maintenance, and upgrades.

IN THIS WHITE PAPER
This IDC white paper discusses the growing complexity of SAP enterprise resource planning (ERP) and the challenge of effectively testing multiple system customizations and changes. It discusses traditional approaches and the need for automated testing and analysis solutions to manage change processes throughout the application life cycle, especially when upgrading to a new version.

Introduction: The Growing Complexity of ERP Systems

Enterprises today are reliant on their ERP to streamline critical business activities. In accordance, ERP systems have become increasingly sophisticated and include a wide range of capabilities, serving multiple departments within an organization through various modules and functionalities.

On top of this already complicated infrastructure, ERP systems provide rich customization capabilities, which enable organizations to change system configurations to address specific business needs and procedures, as well as to reflect frequent changes in their business, competitive, and regulatory environments. These customization capabilities are one of core strengths of SAP's ERP, the long-time market leader. SAP ERP offers almost unlimited options for fine tuning a system by setting and changing parameters, replacing standard ABAP (SAP's proprietary programming language) modules with adapted code variants, and utilizing other built-in techniques.

In light of their flexibility, SAP ERP deployments are highly dynamic. As seen in Figure 1, organizations spend significant portions of their ERP budgets on system updates and changes, including custom modifications, integration with other solutions, and the addition of new modules or packaged upgrades.

Figure 1: Western Europe: Investment Plans for ERP Solutions - Incremental Investments Over the Next Twelve Months (% of Companies)

SOURCE: IDC, 2008

The Upgrade Challenge
The downside of the customization capabilities, however, is the ever-increasing complexity of SAP ERP. This problem becomes particularly apparent in the case of upgrading to a new version. In this situation, organizations are required to make sure that the multiple customizations and changes made throughout the years will operate properly with the new version.

Today, many organizations are in different stages of upgrading to SAP ERP 6.0. According to IDC research that looked into the reasons for investing in new enterprise applications, by choosing to upgrade or replace their ERP systems in order to reduce IT overheads and eliminate manual processes, organizations can leverage the latest tools to save time and resources for all stakeholders. In the case of SAP ERP, after releasing SAP ERP 6.0 and a series of enhancement packages that complement its financial accounting applications, SAP has extended its reach into areas that used to be covered by third-party products or point-to-point business process integration.

These extended capabilities are driving organizations to undertake upgrades. In 2007, SAP saw 5,100 customers running its SAP ERP 6.0, and it has been adding 500 customers every month. By the end of the third quarter of 2008, that figure had reached 8,260, or 28% of its total base of ERP customers of 25,200. SAP executives have indicated that they expect almost all of these ERP customers to upgrade to ERP 6.0 by the end of this decade.

Functional requirements are not the sole reason for upgrades; even more common is the end of current maintenance licensing, which serves as a major driver for SAP ERP technical upgrades. This type of upgrade does not include any functional changes or the addition of new functionalities (which are enabled in functional upgrades). Instead, it offers a rapid, low-cost, and low-risk alternative that is aimed at retaining existing functionalities.

While upgrading an SAP ERP system brings many potential benefits, organizations often cite various major challenges, which mainly relate to the planning phases of upgrading. For the most part, these challenges revolve around budget issues, especially the inability to accurately predict cost and effort, and the difficulty of justifying the project budget. Also cited are operational challenges, such as an inability to evaluate the impact on an existing solution and effectively test the system.

In fact, budget and operational concerns are closely aligned: If errors are not detected in time, rectifying them may result in system downtime and disruption of business processes and may require significant financial and human capital resources. In some cases, undetected errors may also result in regulatory compliance violations, security breaches, and other operational risks.

In order to avoid these pitfalls, organizations need to perform comprehensive tests before going live. However, due the system complexity, tracking and analyzing the multiple customizations and modifications typically turns into a tedious cycle of testing, identifying problems, fixing them, and repeating the process until no errors occur. This is one of the main reasons that the timelines and costs of SAP ERP upgrade projects are difficult to predict.

It should also be noted that, in addition to a complete system upgrade, various other cases exist in which ERP users are required to test changes made in their systems (including customizations), including the following:

• Ongoing maintenance and support operations
• Consolidation of systems due to IT improvements, cost reductions, or mergers and acquisitions
• Deployment of vendor rectifications, support packs, patches, and updates
• ERP application migration
• Changes in application environment due to integration projects, deployment of new technologies and applications, and others
• Deployment of new ERP modules
• Changes in business models and business processes

In all these cases, given the growing complexity of the code base and the multiple code dependencies, a single change in one part of the system may have a significant impact on other parts and may potentially disrupt critical business processes. Hence it is becoming increasingly important to constantly test the entire system throughout its life cycle – from development to configuration changes, maintenance, and upgrades.

Traditional Approaches to ERP Change Management and Their Drawbacks 

Manual Testing
Traditionally, in order to tackle change management issues in ERP systems in general and in cases of system upgrades in particular, organizations have been using a manual trial-and-error approach. Typically, this method is based on creating documented test procedures that specify what to test and how to perform the tests. Detected errors are then reported and fixed. The entire process is repeated until all errors are resolved.

To ease the burden involved in manual testing, a common methodology is to install the upgrade on a copy of the production environment – a "sandbox" system. Teams of ABAP experts and testers are then assigned to perform tests on the sandbox system without affecting the production environment in order to identify and address potential problems before going live.

The manual approach has inherent weaknesses, such as a reliance on the technical capabilities of the testers, which may affect the consistency of the quality of the process. Still, at least in theory, manually testing all system files and objects in order to identify which custom SAP programs will malfunction as a result of an upgrade should provide an adequate assessment. This approach, however, is impractical because a typical SAP ERP implementation may include tens of thousands of standard programs and custom programs. Even if testing a single program takes only a few minutes, testing the entire system may still take weeks if not more.

To shorten the process, organizations usually choose to test only those programs, functions, and processes that appear to have the most impact on key business processes. Thus they face the risk of undetected errors.

Automated Testing Tools
Anther option is to use automated testing solutions, such as regression testing tools. These tools enable retesting applications while they are running in either simulation or production by recording system actions into a script, which can be replayed multiple times to identify errors. Regression testing is typically used to ensure that errors have been rectifying and that code changes have not caused other errors. However, given the multiple customizations and changes in SAP ERP systems, which require this process to be repeated multiple times, regression testing is becoming increasingly impractical for continuous, timely detection of errors.

In addition, some tools provided by SAP, as well as some from system integrators and third party vendors, enable differences to be identified between different SAP systems by comparing existing system configurations and customizations with the upgraded system and pinpointing changes to objects. However, these tools operate at the code level and are therefore less effective for predicting which functionalities would not work upon upgrading or for prioritizing testing tasks based on the impact on business processes.

The Need for Automated ERP Change Management
Another common drawback of traditional testing approaches is limited reusability, which requires users to repeat the testing process each time from scratch. Ideally, SAP ERP users should test the system each time a major change occurs. However, in practice, testing is seldom performed due to the extensive use of resources it necessitates.

The inability to effectively and frequently test the system results in increasing exposure to system malfunctions and errors. Rapid pace of changes exacerbates this problem. Furthermore, due the limited coverage of traditional testing methodologies, organizations often rely on assumptions regarding suspected faults that may turn to be incorrect or to have less impact than predicted, resulting in unnecessary efforts and delays.

As mentioned, the ineffectiveness of traditional testing methodologies is one of the main reasons that ERP projects take longer than planned and go over budget: According to IDC data, 37% of ERP projects run over the estimated implementation time, while 17% of ERP projects run over budget. In times of economic uncertainty, these statistics may cause organizations to postpone their upgrade plans. In accordance, recent IDC research found that sales of new licenses and upgrades for ERP applications are down, which is due to clients looking more deeply into "keep the lights on" type solutions.

To address the shortcoming of traditional approaches to change management in ERP systems in general and SAP ERP in particular, there is a need for new solutions to facilitate and automate various processes throughout the different stages of the system lifecycle. More specifically, an automated ERP change management solution should be based on the following capabilities:

• Identifying customizations and other changes (e.g., deployment of patches, support packs, and new modules)
• Identifying the most critical business processes, transactions, and functionalities supported by the system
• Analyzing the impact of change at both the code and business process levels in the planning stage
• Defining the scope of testing and testing priorities
• Analyzing which functionalities and customizations are being used and which are obsolete
• Automating the process of synchronizing changes across development, testing, and production landscapes
• Enabling automatic rollback of changes to previous versions

By providing these capabilities, an automated life-cycle management approach for ERP systems can eliminate much of the guesswork and the resource-intensive manual processes usually involved in the planning and execution of ongoing customization changes and system upgrades. Furthermore, it can enable organizations to significantly reduce the upgrade costs and more accurately plan the upgrade budget and scope.

PANAYA ERP CHANGE MANAGEMENT
Panaya is a provider of automated software-as-a-service (SaaS)-based solutions for managing change processes in SAP ERP systems throughout the entire system life cycle, focusing on upgrade and support automation. The company's solutions identify which SAP modules and related processes will be affected by customizations and changes and analyze their impact before testing or implementing them in the production environment.

Panaya's solutions operate by initially collecting all the code and usage logs from the customer's system. The data is then transferred to Panaya's facilities for the purpose of running an automated simulation of the current, unchanged, code on the new version of the SAP ERP software, covering all the processes and multiple potential scenarios the system might experience.

Using code crawlers, customization changes in each program can be identified and analyzed to detect instances of malfunction, exception, missing parameters, truncation, and other errors. Cloned programs (copies of standard SAP programs that are used for modifications), which are typically difficult to track and may cause different problems upon upgrade, can also be identified and handled. This way, the number of false positives that are usually identified using traditional testing methods can be significantly reduced.
Customization change impact analysis results are stored in a central repository, which includes dependency mapping of the system configurations. This correlates each change to each affected program and related outputs.

Using a Web-based interface, ERP professionals can report planned changes to the production system and receive analysis of the related potential impact on the entire system. The results are ranked, personalized, and expressed in business language and can be presented in a number of ways to provide users with a clear picture of the impact of customization changes. For example, affected programs can be grouped by their related ERP modules, by similar impacts, or by the effect on specific programs.

Based on this technology, Panaya offers the following solutions:

SAP Upgrade Automation: This automatically analyzes the ABAP code and the system configuration to provide users with inputs such as potential errors and how they can be rectified, which modules and transactions are at the most risk and need be tested, and others. In addition, the solution can identify custom programs that are modified copies of standard SAP processes and can therefore be eliminated to reduce system complexity. Other capabilities include detection of authorization conflicts due to changes in privileges and role definitions, detection of unnecessary modifications that can be replaced by similar functionalities in SAP ERP 6.0, and progress tracking of the upgrade project.

SAP Support Automation: This analyzes functional changes to the SAP ERP system and support packages before they are released. The solution evaluates the code interdependencies in the system and automatically creates test plans, classifies changes based on levels of risk, and alerts managers and other stakeholders to potential risk to related SAP processes that affect their operations.

In addition, Panaya provides the ABAP Code Cleansing solution, which enables the size of the custom ABAP code base to be reduced by finding and eliminating unused programs and objects, cloned programs, and modifications. Using this solution, SAP ERP customers can shorten and simplify the implementation of system upgrades, support and enhancement packages, Unicode migrations, consolidation projects, and regulatory-driven changes, as well as routine maintenance tasks.In addition, Panaya provides the ABAP Code Cleansing solution, which enables the size of the custom ABAP code base to be reduced by finding and eliminating unused programs and objects, cloned programs, and modifications. Using this solution, SAP ERP customers can shorten and simplify the implementation of system upgrades, support and enhancement packages, Unicode migrations, consolidation projects, and regulatory-driven changes, as well as routine maintenance tasks.

Panaya uses the SaaS delivery model, offering potential clients the possibility to outsource the computing-intensive analysis process. As no software installations are required on the customer side, the company's solutions can be set up immediately, enabling organizations to reduce associated risks and costs.
Panaya has made significant efforts to secure its SaaS infrastructure and maintain the privacy of sensitive customer data, including the use of encrypted communication and user authentication solutions in order to prevent unauthorized access and activities. In addition, it maintains separation between the offline servers in which the customer code is analyzed and the online environment in which the change impact analysis results are stored.

CHALLENGES/OPPORTUNITIES
Panaya tackles the key challenges encountered in SAP ERP upgrades and other change management issues – the inability to accurately predict the errors that will occur as a result of an upgrade. Traditionally, organizations have addressed this issue using a trial-and-error approach, which requires lengthy and cumbersome manual testing processes. The inefficiency of this testing method is one of the main reasons that upgrade projects often result in mounting costs, delays, and system outages due to undetected errors.

By simulating an SAP upgrade, Panaya enables customizations and potential conflicts to be identified and their impacts on the entire system's behavior to be analyzed and provides rectification recommendations before going live. Organizations can thus create more accurate budgets and project plans by pinpointing the required development and testing tasks. According to Panaya, its customers have been able to cut their testing and planning times by 50% while significantly reducing costs and risks. The company's solutions are SaaS-based and require no software installations, thus enabling fast setup.

These capabilities place Panaya in a good position to capitalize on the market opportunity presented by the steady stream of customers upgrading to SAP ERP 6.0. Although SAP ERP upgrades are typically high priority, the economic crisis makes SAP users more reluctant to take on projects that require lengthy implementations and high costs. As Panaya's core value proposition revolves around facilitating, shortening, and reducing the costs of upgrades, it may be highly appealing to many SAP ERP users.

Looking ahead, Panaya should consider developing a more general change management framework that can address change impact analysis needs in heterogeneous application environments. This means that the company should build similar capabilities for other leading ERP suites (most notably Oracle) and, in the future, other types of applications. Panaya should also consider expanding its capabilities to enable not only the automatic identification of potential failures but also the automatic rectification of them. Although this requires significant research and development efforts, it is likely that this functionality would be highly beneficial to ERP users, as it may further reduce the time and cost of upgrade projects.

Panaya offers a fundamental shift in the way organizations are managing change processes and upgrades of critical business applications such as SAP ERP. Traditional testing methods that are based on cumbersome and inefficient manual processes are one of the main reasons that upgrade projects often result in mounting costs, delays, and system outages due to undetected errors. Panaya addresses those concerns by automatically identifying potential errors in custom programs and prioritizing what must be tested and what does not need to be tested. It thus enables organizations to estimate project budgets and timeframes accurately, eliminating the need for lengthy comprehensive manual testing.