The need for efficient software development has taken on greater importance as enterprises introduce more and more digital services and add automation capabilities to enhance business processes. Managing software projects might not be at the top of CIOs’ priority lists, but it is something that IT leaders will have to master.

There are plenty of challenges involved in managing software projects, and IT executives who learn how to address these hurdles can help their organizations build better applications to drive business growth and enhance customer experience.

Here are some of the more likely challenges IT leaders and teams face with software projects, and how they can address them.

Delivering on time and on budget

Completing software projects in a timely manner while staying within

budget is a long-time challenge of software development. Any number of things can happen to cause delays and drive up costs.

One possible solution is to embrace the agile methodology of software development. Agile calls for collaboration among cross-functional teams as well as end users, and promotes adaptive planning, evolutionary development, continual improvement, flexibility in responding to changes in requirements and early delivery of products.

“Agile software development projects iterate the cycle of plan, do, check, adjust — and the end user or representative sponsoring [the project] is key in all these stages,” says Ola Chowning, a partner with global technology research and advisory firm ISG.

The waterfall method of gathering all the requirements, designing the entire software capability to meet all the requirements, building all the needed capabilities and reviewing and obtaining buy-in from end users is rarely used today, Chowning says. “This older method, by the way, is where on-time, on-budget challenges were the most onerous, because of the guessing game created when the software team had to estimate large bodies of work and assume some level of acceptance or rework by end users,” she says.

Agile enables the software team and end users to “collectively learn to plan better, work better and adjust more quickly, and outcomes become far more predictable as the way the team works becomes more predictable,” Chowning says. “On time and on budget are much easier to judge with the finger on the pulse of expectations of both the users and the developers.”

Creating and maintaining an agile culture

While adopting agile makes sense for software development at many organizations, it can come with hurdles. And many IT leaders who think their organizations have instituted agile practices fail to understand that what their teams are undertaking isn’t, in fact, agile.

“The intersection of agile software development practices and ‘traditional’ project management remains a challenge for many organizations,” Chowning says. “By now, you would think we would have cracked this nut, but it still seems to stymie many of our clients.”

Whereas software development is approached with a strictly agile way of working — repeated sprints, stories, and multiple releases that build up the end software product in an iterative manner — many organizations continue to struggle as they attempt to manage projects in a waterfall manner, Chowning says.

“This often begins during the project’s business case for funding, where we are typically asked to estimate the total outlay to be reported against in a traditional waterfall-phased framework of requirements, design, build, test and deploy,” Chowning says.

More mature organizations are turning to a project management approach that instead lays out the estimate of overall cost to overall value in a more steady approach across time, Chowning says. “Those who are using this agile project management approach are able to reap some of the real key benefits of agile, [but] it may require some adjustment of investment decisions or even financial practices in terms of project spend.”

A huge challenge for IT organizations is driving the agile model at the enterprise level, says Christian Kelly, managing director at technology consulting and services firm Accenture. “Agile at the team level is now widespread, but recent data suggest that it’s not going as well at the enterprise level, as most organizations struggle to connect strategies to the work their teams are doing,” he says.

This limits organizations’ ability to prioritize portfolios, plan for capacity, manage dependencies and connect goals to outcomes, Kelly says. “To deliver on the promise of agile, organizations need to implement the agile culture, systems, and best-in-class tools needed to better connect strategies to outcomes,” he says.

Aligning projects with overall organizational goals

“IT projects cannot be done in a bubble,” says Chetna Mahajan, chief digital and information officer at analytics platform provider Amplitude. “If your initiative is not aligned with business priorities, you are not set up for success from the outset and you will be swimming against the current at all times.”

To ensure business alignment and buy-in, all software projects should have a business executive sponsor, Mahajan says. When her previous company was implementing configure, price, quote (CPQ) software, the executive sponsors included Mahajan and the chief revenue office.

“This provided us with an escalation channel for both business and technical decisions and deliverables,” Mahajan says. “It was no longer perceived as a technology initiative and it got the visibility and attention it needed across the company. We not only came in under budget and on time, but also were able to increase automation 30% and reduce sales cycle by a couple weeks.”

Most technology projects fail because they lack concrete key performance indicators (KPIs), Mahajan says. “I categorize project metrics mainly into two buckets, one that monitors project execution and the other that measures business outcome,” she says. “What we can’t measure we can’t improve. While it is important to stay the course on budget, scope, and timeline, we must keep a constant eye on the business KPIs.”

The KPIs for a project should be specific and linked to company goals. “This not only helps create a culture of accountability, but also allows for companies to validate their business case to inform future investment decisions,” Mahajan says.

Winning over stakeholders and sponsors

Culture is often a key challenge in the ability to manage software projects in an agile fashion, Chowning says, because sponsors and key stakeholders of the project need to be comfortable and willing to work in the manner most suited to agile.

“Many may still want, instead, to try to work in a more traditional manner—build all requirements, design the entire end state, and only then build and deploy the entire end state,” Chowning says. “This can present a dilemma, as the software development practice and the project management practice try to proceed in two completely different and disconnected approaches.”

Educating the sponsors and key stakeholders in an agile project management approach, and helping them adjust behaviors, is key to managing expectations and enabling software development to proceed in the most effective and efficient manner, Chowning says.

It’s important to engage user representation up front and then continually throughout the iterations of the software development, regardless of the methodology being used, Chowning says.

“Gone are the days when it is sufficient to talk to users up front, and then not engage them again until some mystical user acceptance testing towards the end of the project,” Chowning says. “Users, or user representation [should] be engaged in all aspects of the software development and designs. Small feature developments, prototypes, trials and showcases are all useful means of ensuring users are both engaged and feedback is obtained constantly.”

Need for new development — and management — skills

“One of the biggest challenges we face [is] how to ensure we are continuously providing a strong developer experience and managing the ongoing upskilling of our employees as technologies evolve,” says Amit Sharma, CTO at financial technology company Broadridge Financial Solutions and former CIO at financial services provider Western Union.

“This means creating [automated] solutions, providing a secure, stable environment to develop and test, and equipping our developers with a suite of tools that facilitates a simple, manageable experience and alleviates the overhead and burden of heavy administration,” Sharma says.

With the rapid pace of change in software development, companies need to  train software engineers and others to adapt to new technologies, languages, and development processes. IT and product leaders need to acknowledge that there might be interruptions in projects because of the need to develop new skill sets, Sharma says. They also need to value the people involved in software development.

“It is critical that we recognize that our technology teams have built solutions over the course of many years, and as a result have become the subject matter experts not just of the system, but of the product as well,” Sharma says. “It is [vital] to bring them with us into the next generation of our product, no matter the technology it is founded on.”

In addition to the need for new developer skills, many IT leaders need to hone their own skills.

“Many IT leaders suffer from a massive talent gap in the ability to understand user needs, to create software roadmaps that meet business needs, to drive trade-offs against these roadmaps, and to move from process-based thinking to customer value and customer journey thinking,” Accenture’s Kelly says. “This is why concepts like value stream mapping, customer jobs/value propositions, and design thinking have become so important.”