Highly Competitive - software industry insights

The Resilience of Master Data Management

Final article of a six part series on the "New Age of Master Data Management"

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

For years there have been industry professionals and analysts who seemingly find pleasure in declaring some type of technology as "dead". This compulsion has been a constant in the software industry. Master data management (MDM) is no exception. Some have been convinced that MDM was DOA practically from the day it emerged as a distinct form of data management and data governance. Making such a declaration attempts to make the case that MDM no longer serves a purpose or that it never succeeded in its 'mission'.

The proof of what MDM can do to transform an organization is easy to find. Numerous case studies show how MDM successfully improved data management and data quality, took companies to greater use of data, and contributed to real value in support of goals and desired outcomes. Beyond case studies, the vendor landscape for MDM software solutions continues to grow and evolve. Both traditional and newer MDM software vendors have developed innovative solutions and approaches to master data management that offer improved alignment with business requirements and objectives, and faster paths to realizing value.

If MDM is so useless, why do we have all this activity over these many years? Why would new and innovative technologies continue to emerge continuously if organizations weren't finding that MDM helps them meet many data, process and business challenges?

 

If not MDM, Then What?

I'd hope by now that most organizations understand that they live and breathe data for just about every aspect of the business. And if these organizations want to at least ensure competent use of their data, they should also understand that data management is a must-have. But today companies have to do a great deal more with the avalanche of data that flows into, through and out of the organization. Data management moves several levels higher for meeting business needs when MDM and data governance initiatives are implemented. MDM practices and processes coupled with the right-fit technologies enable many business-critical objectives and contribute to competitive edge.

Those who denigrate MDM seem to imply that MDM serves no purpose. Without MDM, an organization will likely lapse into a sort of 'data anarchy' where there is no common standard for data and definitions, no confidence in the reliability and timeliness of data, no optimization of the systems and processes that run through every business function and activity. Such 'anarchy' can even infect the selection of technologies to handle data, due to the lack of an overarching framework based on corporate data strategies that would have been developed through MDM initiatives.

Organizations will always have to manage business data. But that's not enough. MDM introduces higher quality processes that are clearly aligned with business needs and objectives. Organizations benefit from master data that provides context for other sources of data to be able to achieve optimal results from comprehensive analytics, insights and improved decision-making. Master data processes can help identify missing data, inaccuracies and redundancies that will likely skew analytics results with potentially harmful consequences.

 

Unrealistic Expectations for MDM

If organizations have weak or no strategies around data usage for business, MDM initiatives and technologies can't fix what's already broken in organizations. MDM practices and processes combined with technologies do not deliver a 'magic wand' that will conquer all organizational ills. So-called MDM 'failures' have much more to do with difficulties and obstacles emanating from the politics and people aspects that are so critical to successful implementations of MDM and data governance initiatives.

For MDM to deliver value and benefits, management (often including the C-level) must have good comprehension of the important role that MDM plays across the organization. And management must deliver full backing of what it will take to successfully implement comprehensive MDM. There is now extensive guidance available through publications, blogs, consultants, and many other resources that show how to connect MDM to business value through means such as strategic business cases, and metrics that link MDM activities to desired business outcomes.

A well-implemented MDM initiative actually strengthens the strategic nature of organizations since such an initiative is clearly connected to business-drivers and to helping to achieve goals and objectives. The primary leaders of MDM-related activities should work to align managers across the organization as supporters of MDM. The benefits to each functional area should be clearly communicated, along with the strategic vision of how MDM contributes to specific business impact.

 

Why MDM Has Persisted

MDM provides a core platform of advanced data management and data governance capabilities that are essential for organizations building strategies around data and analytics. After all, master data comprises the consistent business-ready data that runs through many (if not all) processes, systems and applications. As data-focused organizations continue to evolve the use of data and analytics, MDM grows in significance and innovative substance. Master data has become the "rosetta stone" for highly disparate sources of data. When big data is connected to master data that provides context and relationships, organizations can now derive value and usability from big data.

MDM practices and technologies, and the development of reliable master data, are not the end objective, but are the means to enable organizations to realize strategic vision and achieve key objectives. MDM initiatives achieve success because they focus on specific business drivers and business-led initiatives, and are set up to change as organizations change. Such MDM success translates into organizations that continuously benefit from faster utilization of timely, reliable data.

MDM anchors many significant, even critical, functions:

• Real-time analytics / BI processes
• Big data analytics
• Automated processes for information and data processing applications
• AI and machine learning efforts
• Compliance and risk management 
• Improving customer experiences and interactions
• Cost reductions
• Ability to anticipate, analyze and adapt to market changes
• Agile responses to change for competitive edge

MDM has persisted because of what it tackles. MDM jumps into many of the messiest aspects of an organization: data silos, unconnected systems and processes, management turf wars, politics, and the organizational roadblocks to disparate people working together effectively for common purpose. What MDM takes on to achieve business-usable master data and better processes reflect many of the problems that organizations must solve to survive, adapt, compete, and achieve desired outcomes.

So it's about time for master data management to enjoy a 'New Age'. The creation and use of master data for business purposes has always been a good idea. The goal has been to make sense of all data that touches the organization: clean it up, add context, enrich it, and put it to work strategically as well as operationally.

 

Organizational Resilience in the Face of Business and Market Volatility

Many organizations continue to encounter increasingly volatile markets that have given rise to more complex business environments. Such volatility can become a hurdle for achieving financial success and often translates to significant risk. Organizations need more sophisticated analytics and decision-making processes to assess market conditions and to determine whether "risk" means opportunity, danger, or even both. Master data is an important component of essential processes for insight into market conditions, supporting functions like real-time continuous analytics and responsive decision-making.

While more senior executives do see the value of data analytics and MDM, many organizations are still making critical decisions based on spreadsheets. These organizations aren't able to respond to market volatility with comprehensive intelligence rooted in the right data and analytics. Since MDM is tightly bound to business drivers and functions, it's already positioned correctly as the right partner for real-time analytics that support critical business decisions.

Organizational agility and adaptability are defining facets of competitive edge, particularly in the face of market volatility. The agile business requires continuous processing of data from many disparate sources that must be transformed into consistent and trustworthy data with relevant context. Speed to decisions and to actions of change is improved and better grounded in the right information when MDM processes are involved.

MDM can provide pathways to help 'future-proof' organizations. Being future-proof means harmonizing technology, people, and processes to tackle the challenges ahead. When implemented effectively, MDM greatly improves methods used to address the challenges that organizations face every day. MDM practices, approaches and technologies continue to evolve to help solve complex business problems.

 

The New Age of MDM

Master data management and data governance have indeed entered the 'New Age of MDM'. The practice of MDM has become stronger through building on common cause with business, rather than a sole focus on technology. Technology vendors have made advancements with MDM software solutions by understanding the critical nature of MDM aligned with business needs, objectives, optimal processes, and involved business users.

MDM provides the greatest benefits when practitioners embrace the strategic aspects of an organization and help to enact them through processes, practices, policies, and organizational changes. Strategic use of MDM also gives rise to cross-functional and cross-organization integrations of people and systems. A successful MDM initiative comprises a holistic endeavor to provide strategic organizational and infrastructure change, as well as manage complex data processing. Part of MDM holism is the intertwining of business, people and technology in the most effective mix possible.

On the software solution side, MDM is the "poster child" of the critical need for business strategy to 'meet and mingle' with technology strategy. In a sense MDM technology is a conundrum. The technology should take a back seat to all of the non-technical activities, objectives and requirements for MDM implementations. But technology can also be a game-changing component of MDM initiatives.

Now, on to the 'Next Age of MDM'!

 

Image source: nicholaspetrie.com

 

About the author: Julie is an independent consultant and industry analyst for B2B software solutions, providing services to vendors to improve strategies for customers and target markets, products and solutions, and future direction. Julie has expertise in several solution spaces including: data integration and data quality; analytics and BI; business process, workflow and collaboration; digital marketing, WCM and social media; and the pivotal importance of user and customer experiences. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

 

MDM for Business Value: New Approaches

 

Part 2 of a six-part series on the "New Age of Master Data Management"

 

Master data management is entering an 'evolutionary' era where MDM implementations better mirror how businesses use data for numerous purposes, and where people, practices and processes come to the forefront as essential elements. MDM functioned as yet another solution space that was harmed by disproportionate attention to its underlying technologies. But MDM has always been far more about business objectives and needs, as well as the problems to be solved to achieve them. Along with a more overt "de-emphasis" on technologies, we are now seeing more vendors enabling the critical integration of technologies, people, practices and processes with near term and long term business objectives, to achieve the successful utilization of MDM programs.

A major change to MDM approaches calls for combining technology and non-technology paradigms, to reflect the new reality that the business world isn't just "relational". Relationship (network / graph) data continues to grow in importance – but traditional row-and-column databases are inadequate to properly handle such data. MDM approaches and technologies must now process complex connections, hierarchies and relationships between multiple entities / domains. Organizations want to go beyond traditional transactional data to capture data that describes behavioral aspects that can amplify context.

 

 

 

 

Out with the Old

Over the years, organizations often took wrong turns with MDM by relying on problematic approaches like "big bang" or "bottom up". Either way has usually resulted in big problems and poor outcomes. And both of these approaches have had a history of focusing too much on IT driving the MDM initiative, while failing to incorporate the voices of essential business stakeholders. These traditional approaches usually take too much time, carry too much risk, and end up being too expensive -- often not even achieving essential objectives.

Successful MDM projects require strategic vision that connects to current and future business needs and objectives. But too often organizations have decided that 'strategic vision' means tackling a large scope of work ("big bang"), instead of fleshing out the right strategies before starting any work. Organizations that attempt to execute an entire MDM strategy at once encounter significant obstacles including serious cost overruns, project management turmoil, and chaotic use of IT resources.

The "bottom up" approach relies on IT and / or lower level managers to drive the MDM effort, while disregarding vital upper management and business stakeholders. Often disconnected teams work in silos, but are expected to somehow arrive at the same "desired end state". This introduces the near-impossible task of trying to orchestrate non-collaborative work.

 

New Approaches Abound

Obviously new approaches for MDM are made better, or even possible, because of newer technologies. But MDM technology by itself will fail to achieve what businesses need. Of primary importance is the work done to establish the right vision and conceptual framework for what master data can accomplish for the business. Organizations need to develop hierarchies of strategies, plan processes and practices, make decisions for design and modeling – among the many preparative tasks that usually ensure more effective utilization of MDM. Then organizations can build out from there, ultimately considering which technologies should be implemented.

Offerings from new MDM vendors and fresh approaches by established vendors are responding to what organizations need and want from master data: centralized reliable data that reflects how organizations work and the kinds of data they need, increasingly in real time contexts. MDM is moving towards faster implementations, broader capabilities in a single extensible platform, and the ability to work with any kind of data.

 

Organizations are encouraged to explore new MDM approaches and software solutions that are designed to overcome the traditional challenges that have plagued MDM initiatives and that will better fit on-going business objectives, needs and functions:

 

The Cloud and MDM-as-a-Service

Obviously, SaaS and cloud services have distinctly influenced how customers buy and access software, and how most software solutions are now deployed. Master data management is no exception. As with other software solutions moving to the cloud, MDM can become more user-oriented and fully focused on the real purpose of MDM, without a lot of the technology "baggage" that comes with on premises solutions. Better partnerships between IT and business roles can be forged to optimize MDM processes and how they contribute to real business needs and value.

There are still some obstacles to fully delivering MDM-as-a-Service to business users. But, as with other data management solutions that have moved to the cloud, a great deal of progress has been made. Self-service models contribute to the expansion of different kinds of access as appropriate for different kinds of users. For example, a number of cloud integration platforms provide seemingly simple graphical interfaces that can accelerate usage development and ensure better runtime management. Even technology-adept data management experts want self-service access to MDM solutions. Point-and-click graphical interfaces accelerate all aspects of creating and maintaining MDM processes and applications.

 

Business-centric Platforms to Solve Business Problems Faster

New and established MDM vendors are working to revamp the MDM design approach. One business-centric approach is based on starting with the desired end state, and all that is required to define it. Then teams work backwards from the desired end state in a reverse-engineering process to figure out how to attain it. The MDM solution platform helps the organization create the models that are needed for a particular end result, and then automatically generates the processes that will carry out the appropriate MDM tasks as extensions of the models that were created. Most "end results" are sharply focused on business needs and value, particularly since extensive business knowledge is required to accelerate development of the resulting MDM processes.

Often this approach brings about a shorter, automated and rule-driven lifecycle that means more iterative development and much shorter implementation times. Business and IT teams can work in parallel, while utilizing the MDM platform to orchestrate and synchronize their efforts. A further extension of such business-oriented platforms leads to simplifying the development of industry-specific business applications that take advantage of master data to solve business problems.

 

Multi-Dimensional Data – Complex Relationships and Hierarchies

Contextual MDM solutions handle new approaches for multi-dimensional and complex hierarchical data, including social and commercial graphs that underlie business use cases such as customer-responsive interactions across digital channels. Such solutions allow unlimited combinations of categories, context and definitions. Multi-dimensional often means multi-view, reflecting the need to view domains or entities (such as customers) in different ways, in order to derive better insights from data. Today, entities extend beyond the traditional customer and product pairing to include items such as asset, location, supplier, finance, and personnel data.

To better serve business needs, contextual master data management can provide a consistent and multidimensional profile across data and business silos. To meet this imperative, the emphasis falls on the orchestration of master data and processes, to customize a trusted, relevant master view aligned with data consumption needs. The growing interest in multi-dimensional models of master data reflects the huge growth in data elements that are needed by most organizations, as well as increased dimensionality and deeper levels of data.

Contextual MDM utilizes graph databases for additional flexibility in managing multiple domains as "one", instead of creating separate domain models. This approach takes MDM implementations from systems to views. Graph databases can help organizations use master data in more analytic and insightful ways, as well as take of advantage of real-time graph analytics.

 

Rise of Analytical MDM

With the increase in high performance computing, particularly in support of advanced analytics, analytical MDM now occupies an important position for many organizations. Vendors are integrating MDM capabilities with analytics platforms, for more immediate insight benefits. Underlying technologies include capabilities for machine learning, processing big data, graph databases, and visualization tools. Applications are extensive: fraud detection (real-time and predictive); real-time customer intelligence and product recommendations; reducing costs for many aspects of healthcare.

Unlike traditional approaches for setting up data warehouses for business intelligence (BI) purposes, analytical MDM can perform all MDM functions to ensure data quality, useful data enrichment and the addition of context through consolidated "views" of customer, product and other data domains. Today's MDM hubs often accommodate up to tens of thousands of data elements for multi-domain approaches. Numerous MDM architectures can manage multi-dimensional data with complex hierarchies. Such data provides the wide array of metadata and attributes that are necessary for real-time business processes, particularly for initiatives focused on customer experience and systems of interaction. Analytical MDM approaches (and technologies) are seen as stronger solutions for these sophisticated business needs.

 

Knowledge-Based MDM – and Taxonomy

An MDM initiative faces a tremendous challenge to align disparate data sources and then perform accurate data quality processes to derive reliable and usable data. Part of the challenge is verifying the validity of data quality results.

Traditional table-driven approaches for data quality needs can be arduous and time-consuming. Organizations have adopted knowledge-based approaches that consolidate information from all records that are accessed in conjunction with an MDM system in order to provide context in terms of customer and other domain-related data. Such knowledge bases often include millions of entries that provide correct and incorrect spellings, relevant patterns, and other data quality attributes. Proponents of knowledge-based MDM claim rapid processing of entire data sets for data quality purposes.

Knowledge graphs bring agility to visualizing master data models and linkages. Conversely, changes can flow into master data and data models through the knowledge graph. Knowledge graphs are built for known needs; which means they can accurately align with business needs. Salient entities and relationships, concepts and content can be incorporated into knowledge graphs.

A frequent partner for knowledge graphs is taxonomy which brings in semantic architecture. Until recently, taxonomy has been used more frequently for enterprise content and information management, and enterprise search. Taxonomy helps MDM data quality processes better handle ambiguity through inherent capabilities for understanding context, resulting in precise meaning. Since many of the underlying processes for MDM and for enterprise taxonomy development are similar, it can be effective to run these efforts concurrently or use one to jumpstart the other.

 

Re-Thinking MDM with Integrated Approaches

One of the most important changes to master data management approaches has grown out of the fact that much of MDM and data governance revolve around business use cases and business users. Everything done with MDM is first and foremost about the business. Ironically - and painfully - the increased involvement of different business roles constitutes a 'new' approach for many organizations.

Approaches that can help the "people issues" of MDM programs include:

• Embedded collaboration practices and technologies – integrating business and technical users to achieve intelligent and optimal usage of MDM
• Business user access and visibility into appropriate MDM processes, including continuous reviews of data and use of tools like data profiling
• Make it easier and faster for everyone to do their share of the work
• Comprehensive and responsive orchestration of many different teams and processes, to work as one

While new approaches have been discussed as individual items in this article, for real world implementations, multiple aspects of different approaches usually come into play for a particular organization, and need to be well integrated for effectiveness. Once again, such integration reflects the complex world of business.

In the past few years, new vendors have entered the MDM space with solutions that have a better focus on what businesses need. New and old vendors are also dismantling the old ways of how MDM has been implemented to more realistically support the diverse array of data sources that businesses need to utilize, particularly for faster, more agile analytics processes. The next article of this series will examine the evolution of MDM technologies.

 

Image source: landarchs.com

 

About the author: Julie is an independent consultant and industry analyst for B2B software solutions, providing services to vendors to improve strategies for customers and target markets, products and solutions, and future direction. Julie has expertise in several solution spaces including: data integration and data quality; analytics and BI; business process, workflow and collaboration; digital marketing, WCM and social media; and the pivotal importance of user and customer experiences. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

 

Master Data Management Rebooted

 

Part 1 of a multi-part series on the "New Age of Master Data Management"

 

The 'New Age' of Master Data Management

It's about time that master data management (MDM) enjoyed a 'new age'. The creation and use of master data for business purposes has always been a good idea. The goal has been to make sense of all data that touches the organization: clean it up, add context, enrich it, and put it to work strategically as well as operationally.

Unfortunately, MDM has been plagued with certain problems from the very start. This stems mostly from the fact that MDM lands hard on some of the messiest aspects of an organization: data silos, unconnected systems and processes, management turf wars, and organizational roadblocks to disparate people working effectively together for common purpose.

 

Why Does Master Data Matter?

As the importance, value and sheer volume of data assets grow, so does the critical need for trustworthy, up-to-date information that is ready-to-go for any purpose. MDM processes consolidate data from numerous sources in meaningful ways. Master data also provides context and reveals data relationships for sources like big data that often exist as fragments without useful referential connections.

Master data has a large and growing role in areas that matter to organizations: strong brand presence, multichannel customer interactions, right-fit content and information, and highly variable buying journeys.

Analytics, particularly of the real-time variety, produce more realistic and trustworthy results when utilizing master data. Master data is often a key part of predictive analytics in support of: supply chain processes; personalization for improved customer experiences; logistics; operational optimization; and safety and preventive procedures.

A core element of MDM is data quality. Improved data quality impacts endless aspects of the business for reasons that should be obvious. But all too often, business management fails to comprehend how much poor data quality impacts profitability, productivity and even perception of the brand. Real world incidents that have been repeated at far too many organizations include:

• Inconsistent data of poor quality has led to extensive quantities of items in warehouses to "disappear" from systems records, never to be sold or consumed by the business. This can amount to millions of dollars of losses.
• Contradictory and confusing product lists and catalogs that deter purchases by customers and make sales staff look stupid – again with no purchases resulting.
• Costly pricing errors that persist undetected over time can have breath-taking negative impact on the bottom line.

 

Roadblocks, Detours, Dead Ends

Over time traditional approaches for MDM implementations have faltered for several reasons, many of which echo the failures of traditional ERP installations:

• Implementations take too long and cost too much.
• Project scope is often much too large.
• Business and IT teams are poorly coordinated for achieving the desired end state.
• Results fail to meet important business needs and / or show value in an obvious and timely fashion.
• Traditional approaches lag in keeping up with new technologies.
• Solutions have delayed including new data sources like the many varieties of big data.

A continuing obstacle is the lack of involvement of upper management with such important activities. If organizations want to tap into more data that is trustworthy and business-ready, it requires buy-in from upper management on down. Such buy-in is shown through clear commitments, upper management champions, and real data strategies – basically understanding the high value of data and what it takes to achieve it.

 

The Reality of MDM: People and Politics

What many people get wrong about MDM is thinking that it's all about the technology. In reality, it's much more about people, processes, and (unfortunately) politics. The people and politics of the organization can throw up significant obstacles to successful MDM programs. This is more likely to happen in organizations where upper level management has failed to provide its leadership and involvement in MDM implementations.

Silos permeate organizations from top to bottom, and not just in terms of software systems or digital assets. Collaboration between people is often limited. Internal politics can bring on secrecy or "turf wars" when it comes to the ownership of data and business processes. Resistance to change, lack of trust, desires to "hoard" data and information all come into play. Orchestration and collaboration of business and technical teams have always been critical to MDM success, but can be derailed by internal silos and politics.

Of course, technology provides an important means to achieve MDM implementations. But many vital activities aren't technical. For MDM and data governance to be successful, distinct changes must take place in organizations to eliminate the many silos that prevent the generation of reliable and up-to-date master data.

Not surprisingly, when things go wrong, organizations want to blame MDM technologies for the failures, instead of facing up to what really ails the organization. However, traditional MDM solutions have had significant limitations and often don't do much to help with the "people problem".

 

Wanted: Improved Approaches

A classic approach to MDM, the "bottom up" method, has not worked well simply because unsynchronized teams are trudging separate long roads to achieve the "desired end state" and often don't ever get there. It's easy to imagine the many ways that this method can be derailed. Instead, businesses benefit most from MDM approaches that work the way the business and its employees need to function to achieve real goals and objectives.

One of the most important changes to MDM approaches has organically grown out of the fact that much of MDM and data governance revolves around business use cases and business users. So it's gratifying to see more overtly business-defined approaches that not only delineate the approach but are enabling changes to the technologies that support them. These business-defined approaches reflect the complex, sophisticated needs of many organizations:

• Working from the desired end state and / or from the requirements of complex business problems to create master data management processes.
• Choosing contextual and analytic MDM solutions over traditional MDM tools that can handle new approaches for multi-dimensional and complex hierarchical data, including social and commercial graphs that underlie business use cases such as customer-responsive interactions across digital channels.
• Utilizing new MDM platforms that simplify the development of industry-focused business applications that take advantage of master data to solve business problems.
• Transforming model-driven approaches to be more flexible and agile in order to work with existing business processes as the starting point (instead of creating new processes), to deliver business value more quickly. IT and business teams can then transition to optimizing various processes for additional value.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

In a real world example, a pharmaceuticals company has utilized a new MDM solution to uncover significant information that improves the success of promoting drugs to individual healthcare organizations. Such information includes:

• Which insurance companies or payers accepted by the healthcare organization actually cover the drugs
• And for those payers, which insurance plans provide coverage
• What contracts for the pharmaceuticals company are already in place
• What other sales teams in the same company are already promoting to the healthcare organization
• Other important influence and decision-making factors that derive from complex hierarchies and networks of relationships

Running through such an analysis based on master data before approaching a healthcare organization helps the sales team determine if any business opportunities exist, and reveals better ways to approach contacts in the organization.

 

Master Data Management Rebooted

In the next five posts, I'll present and analyze what is changing for master data management approaches and solutions, including new technologies, and the evolutionary involvement of business roles in MDM and data governance.

One of the final aspects that I'll explore is this: is it time to quit calling it "master data management" and devise a name that better fits MDM Rebooted?

 

Image Source: hbr.org

 

About the author: Julie is an independent consultant and industry analyst for B2B software solutions, providing services to vendors to improve strategies for customers and target markets, products and solutions, and future direction. Julie has expertise in several solution spaces including: data integration and data quality; analytics and BI; business process, workflow and collaboration; digital marketing, WCM and social media; and the pivotal importance of user and customer experiences. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

 

Data Connectivity in the Web-Enabled World

...

  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

As the use of SaaS and cloud services has grown, the API (Application Programming Interface) has come to the forefront as an important technology component in this web-enabled world. APIs are points of entry or connectivity methods, for integrations between different services, systems or applications. Extensive documentation is made available to assist developers, to quickly and effectively work with an API.

Actually APIs have been around a long time, and have been used in data integration processes when other connectivity was unavailable. Both APIs and data integration have come a long way since then. Pre-built connectors for innumerable data sources and formats comprise 'table stakes' for today's data integration requirements. As such, developers and users can quickly set up API connectivity to simplify the sharing of data between: cloud services and mobile apps, data integration processes and data repositories – and enterprise applications residing anywhere.

 

The Web API Economy

A detrimental side effect of the fast growth of SaaS applications and cloud services has been the propagation of endless data silos in the cloud. Vendors for cloud applications provide APIs to enable users to access that data, which unfortunately results in the propagation of endless proprietary APIs. Mobile apps have added another front for the non-stop creation of APIs. While many APIs are available to third party developers as "open" or public APIs, underneath they are frequently proprietary to each vendor and not necessarily "open source".

Vendors for SaaS and cloud services have been able to find advantage in their APIs, both for their own purposes and to benefit their users, beyond accessing their data. APIs have become an economic factor that is opening the door to third-party innovation and developer communities. When vendors provide APIs, technology partners and other developers can create new applications (data mashups, for example) that extend the value of vendor services. Mobile apps in particular need such mashups to quickly and seamlessly pull in information that often comes from multiple sources.

Vendor companies are taking advantage of the APIs they have created to build their own digital sites, mobile apps and even internal business processes and applications. Slowly tools are appearing that enable business users (with good tech savvy) to work with APIs -- following the lead of emerging self-service options for business users in data integration and analytics. So the API Economy is evolving to help different roles get a lot more from web-enabled applications and services.

 

Web API Management

Numerous software vendors have found opportunities in providing API Management platforms or solutions. At the high level, these solutions support:

• Development, publishing and documentation phases
• Creation of integrations through pre-built connectors
• Various deployment and customer usage activities
• Overall management and monitoring of API events and services
• API life cycle management

The primary customers have been companies that create web APIs, but third-party users and partners also interact through API management platforms. These platforms make APIs available to third parties, through a regulated and secure interface.

 

API Management for All APIs

For all the buzz about the API Economy and extending services, ultimately a lot of the value of APIs is in connecting to, aggregating and integrating data for more innovative business use. What's exciting about the API Economy is that it's making it easier to use APIs, even for less tech-savvy business users. When offerings begin to include access for business users, it means that the tools are more streamlined for in-house developers and third party solution providers – translating into faster time to implementation and more time for innovation.

Somewhat ironically, the developers and third-party users of web APIs are now folding in connectivity to data sources and applications in the "traditional" enterprise tech world as new ways to leverage APIs. The current notion of what's called API Management will likely evolve beyond the initial focus on web APIs and APIs created and maintained by a particular company, to the broader enterprise world of systems.

 

Image source: cedefop

 

This post was brought to you by IBM for MSPs and opinions are my own. To read more on this topic, visit IBM's PivotPoint. Dedicated to providing valuable insight from industry thought leaders, PivotPoint offers expertise to help you develop, differentiate and scale your business.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting

 

 

 

Interoperating Services are the Heart of Cloud Data Integration

 

Most SaaS offerings started with a business user focus, providing applications that zero in on solving specific business problems. It's become an everyday occurrence for businesses of all sizes to migrate many of their needs to SaaS.

There are also SaaS applications that serve as components of a comprehensive IT landscape. But when we look at creating and maintaining the overall technology infrastructure in an organization, a single SaaS application really isn't able to provide full-on IT services. So IT has been finding more usefulness in the world of cloud services that have their roots in supporting IT and developers. IT is able to take advantage of cloud-based platforms, such Infrastructure as a Service, for core technology tools.

Obviously, in terms of data and technology interoperability, SaaS and cloud services must integrate with the entire IT world, much of which is still on-premises. Business processes and data-flows now run from cloud to on premises and back. Businesses need agile technology tools to handle the increasing hybrid infrastructures of many organizations.

The overlapping of data and application integration has been accelerating, particularly with the growth of cloud-based integration solutions. Out of this convergence has emerged the cloud integration platform iPaaS (Integration Platform as a Service). iPaaS is evolving into a new class of technology that supports hybrid infrastructures, and both business and IT users.

IPaaS has its origins in Data Integration as a Service targeting business users, and in data integration solutions for the proliferating silos of data in the cloud. It has been growing into a platform that also offers services for IT and developers, to more deftly handle multiple integration scenarios. iPaaS often functions as an orchestration hub for complex integration processes. And it is increasingly taking on more "traditional" data integration scenarios.

With iPaaS, business users can handle more of the basic integration tasks through self-service data integration, taking the load from IT and data management teams. And IT is becoming more in tune with the self-service economy, by taking advantage of pre-built, reusable services that speed up the creation of more complex data integration processes. Through a services approach to data and application integration, IT can more quickly keep up with business demands.

For the service economy, iPaaS has to be more than just another development platform for the tech savvy. The ultimate outcome of iPaaS should always be the creation of services – iPaaS itself is a means to an end. The services created contribute to iPaaS serving as an orchestration platform that can accomplish a great deal for organizations:

• Continue to grow the IT-Business partnership by using iPaaS as the means for business users to plan, design and execute integration processes; and by using iPaaS as a collaboration platform for IT and Business to improve any and all integration processes to more quickly meet dynamic business needs
• Utilize iPaaS services to streamline management of integrations and API environments from beginning to end
• Create a single environment where numerous technologies converge and interoperate with centralized management
• Engender the growth of transparency and insight for improved operational intelligence

While iPaaS solutions have evolved tremendously over the past few years, there are still miles to go. Several vendors have realized that with the disparate variety of data and business usage for integrations, critical functions like data quality, data governance and master data management must be added as core services for iPaaS. API management is another critical component, particularly since cloud services have a tight relationship to the development and usage of mobile apps that create and consume large volumes of data.

 

Image source: http://www.wunderground.com/wximage/canon30dguy/19

 

This post was brought to you by IBM for MSPs and opinions are my own. To read more on this topic, visit IBM's PivotPoint. Dedicated to providing valuable insight from industry thought leaders, PivotPoint offers expertise to help you develop, differentiate and scale your business.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting

 

Intelligent Transportation Systems for Midsized Cities

 ...

  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

When intelligent transportation systems are discussed, the focus is usually on mega-cities (populations of 10 million or more). But midsized cities (anywhere from 250,000 to 1 million people) face many of the same issues as larger cities, especially regarding forward-thinking planning for transportation systems. In fact, midsized cities are still a significant untapped market for intelligent transportation systems.

Right now transportation system technologies are evolving faster than overall management strategies. Adding to the management 'chaos': the proliferation of massive amounts of data related to transportation systems, including machine-generated big data. For midsized cities, data is a key asset for the planning, control and advancement of intelligent transportation systems.

When intelligent transportation systems take advantage of advanced data analytics, the value that results includes optimized infrastructure management and operations, as well as improved transportation services in terms of variety of modes, safety, reliability, and efficiency. Transportation data analytics also contribute greatly to real-time improvements to traveler / user experiences.

Categories of transportation data include:

• User or traveler behavioral data – survey data, social media sources, video, monitoring
• Travel demand and system use – monitoring data, machine-generated sources
• Socio‐Economic data – land use, demographics, income
• Transportation services – frequently as machine-generated data from sensors and meters
• Traffic management data – primarily machine-generated sources
• Infrastructure – roadways, facilities, vehicle fleets
• Environmental impact – emissions/air quality, local health concerns, other pollution/contaminants

Urban transportation networks frequently exist in an ecosystem of transportation modes and other urban governance entities in order to consolidate overall vision, resources and functions, which can lead to better outcomes for all. Intelligent transportation systems should support data sharing across modes, functions and stakeholders to show the value of collecting all relevant data, as well as being able to deliver data to different data management and analytics structures. And "stakeholders" aren't just humans – machines and automated systems are often around-the-clock consumers of transportation data.

 

Managed Services for Midsized Transportation Systems

I've written before about the differentiation that managed services providers can create by concentrating on specific industries and developing the distinct services that will make a difference to that industry. To provide niche or industry-specific services means developing in-depth expertise for the service offering, both in terms of the industry or niche and the problems that need to be solved.

Managed services can be developed to help midsized cities improve intelligent transportation systems through the use of advanced data analytics. The focus should be on new ways to deliver cost-effective services and solutions through cloud platforms, lower priced data analytics tools, and mobile applications.

Beyond services for the continuous data analytics needed to govern and improve urban transportation, managed services providers can help midsized cities understand how to apply analytics results to achieve significant improvements to transportation systems in multiple areas:

• Personalized interactions with citizens using transportation systems
• New services created more quickly
• Real-time operational responsiveness
• Measuring the effectiveness of current decisions and actions

Such services and solutions should be tailored for midsized cities to keep costs down and provide exactly what these cities need for faster, more effective transportation systems to better serve their citizens. A one-size-fits-all approach rarely works. Managed services providers need to craft core services and implementation templates that can be easily customized for different cities and their transportation systems.

 

Image source: swarco.com

 

This post was brought to you by IBM for MSPs and opinions are my own. To read more on this topic, visit IBM's PivotPoint. Dedicated to providing valuable insight from industry thought leaders, PivotPoint offers expertise to help you develop, differentiate and scale your business.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting

Data Integration Services Evolution

 

 

With the increased usage of data analytics to help solve business problems and improve strategies, data integration has come more to the forefront as a critical component of overall technologies for data-savvy organizations. New approaches and technologies for data integration are coming into play to better meet what organizations need, especially to support increased business user access to data management itself.

Over the past several years, data integration services have continued to grow and evolve, thanks largely to the cloud. Tapping into SaaS approaches, the early entrants were data integration as a service and purpose-built or pre-built data integrations.

 

Data Integration -- as a Service

Data Integration-as-a-service (IaaS) became possible as more organizations made use of SaaS, cloud services and even B2B data exchanges. Business ecosystems began growing in size and complexity thanks to online connectivity, promoting ease-of-access for B2B relationships and interactions. As more data flowed in the cloud, the need and demand for integration picked up momentum. Originally IaaS functioned as an IT-oriented service, mostly in stand-alone instances. But IaaS instances were soon embedded in other cloud services and applications, as well as cloud infrastructure platforms.

The ability to quickly move data to where it is needed must evolve beyond cloud services that only provide basic APIs and web services that still require users to build connections. While leveraging APIs and web services for loosely-coupled integration processes, IaaS rapidly added more sophisticated capabilities such as virtualized data services, governance, REST-ful services, and agile interoperability at multiple levels.

Cloud-based IaaS has since progressed into Integration Platform as a Service (iPaaS). The data integration services evolution now involves comprehensive platforms that leverage cloud services for the creation and execution of integration processes or flows that connect to systems in the cloud and on premises -- within a single entity or across multiple organizations. Services include critical capabilities: data governance and management, metadata discovery, data quality, transformation rules, agile access to data sources – all delivered with automation and ease-of-deployment as important aspects of the overall platform.

 

Purpose-Built Data Integrations

As a more narrow approach to integration, purpose-built data integration is designed to meet a specific business need, with pre-defined capabilities bundled in the solution. Frequently purpose-built data integration comes as an appliance. Purpose-built data integration services in the cloud comprise an evolutionary step for making data integration services available to business users, to initiate integrations between specific sources or applications.

Cloud services for purpose-built integrations usually include data replication, migration, and synchronizations for customer data. Typically integrations involve specific SaaS applications like Salesforce.com and Netsuite as "pre-set" sources and targets.

These services work with focused use cases where business users do not have to make lots of decisions about how the integrations will be accomplished. Wizards are employed to walk business users through the process. However, at minimum, business users must have knowledge of the sources and targets, and basic understanding of the data integration task, to work confidently and accurately through the purpose-built approach.

 

Business Innovation from Data Integration Services

The evolution of data integration services must align dynamically to the business as it changes. Business users require rapid responsiveness to integration needs, through technology that is 'proactively' ready for many use cases that reflect expected and realistic business situations. Innovations for cloud-based data integration services contribute to optimal business performance and to the faster delivery of valuable data that frequently has a short shelf life of relevance.

Data integration services can tap into the rich value of the cloud and how it connects businesses through common services, with business processes providing contextual frameworks. Such data and process agility underpins must-haves for the competitive organization: always-on intelligence, new business opportunities, innovations in products and business models, robust collaborative processes for the ever-growing business ecosystem.

 

Image source: trendmicro.com

 

This post was brought to you by IBM for MSPs and opinions are my own. To read more on this topic, visit IBM's PivotPoint. Dedicated to providing valuable insight from industry thought leaders, PivotPoint offers expertise to help you develop, differentiate and scale your business.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting

 

The Missing Pieces of Master Data Management Services

 

A master data initiative is meant to deliver a unified and well-integrated source of cross-organizational data that is reliable and up-to-date, while eliminating silos and redundancies across the organization. Master data management (MDM) software solutions comprise an essential piece for trustworthy analytics that lead to better decision-making. Master data is also a much-needed accompaniment for big data analytics, providing critical context for big data, and as a means to connect big data analytics to business objectives and value.

We're seeing more offerings for MDM services from IT and solutions consultants, managed services providers, and software vendors – with cloud platforms playing a clear role in these offerings. Third party technology services often result in lower costs and faster time to solution use and to the engendering of business value. Unfortunately providers of MDM services frequently only target IT groups, and, as such, fail to include the very important business stakeholders in a client organization. A technology-only approach to MDM results in significant pieces missing from the puzzle, which will impede how well MDM will work for an organization, particularly to produce business value.

These missing pieces relate to people, organizational change and new processes, which in turn often give rise to internal political issues that must be identified and dealt with. For successful MDM initiatives, technology should take a back seat until thorough understanding is mapped out for the people, practices and processes of the organization. Otherwise there will be a serious impairment of the integrated vision of how a master data initiative impacts the organization and the work that is done to achieve business objectives.

A focus solely on technology ensures that the needs and requirements of the business won't be met, and that the MDM initiative will fail to address essential non-technology elements, such as relationships with key people. Business champions, stakeholders and influencers are all critical to the success of any MDM project.

Data governance is another crucial function that aligns the important elements of people, practices and processes with master data management. Again, the plan for data governance must be in place from the beginning of an MDM initiative. It is an essential apparatus for helping to mitigate the many obstacles that can plague MDM.

Third party services providers have a big opportunity for providing MDM services. But such services must include the business and people sides of the story. Often a neutral outside consultancy or service provider can help achieve the organizational change required for MDM to make a difference for businesses. Service provider insight and experience enables the identification of problems from the beginning and provides useful solutions and guidance for clients to mitigate internal politics.

Services providers must be prepared to develop a well-structured MDM vision for clients that aligns to and realistically supports overall corporate strategies, business unit goals and optimal business operations. Such a vision must address not only how business is done today, but what organizations want to accomplish further down the road. A focus on technology alone cannot achieve this sort of visionary work. By providing the full scope of work for MDM initiatives, a service provider will stand well above those who only focus on MDM as a technology "project".

 

Image source: http://www.danpontefract.com/give-piece-a-chance/

 

This post was brought to you by IBM for MSPs and opinions are my own. To read more on this topic, visit IBM's PivotPoint. Dedicated to providing valuable insight from industry thought leaders, PivotPoint offers expertise to help you develop, differentiate and scale your business.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting

 

Meet Your New Cybersecurity Partner: The CFO

 

 

As with other data management and analytics processes, Chief Financial Officers are increasingly becoming an integral part of the management of cybersecurity initiatives. This is primarily due to the huge financial impact that cyber attacks and data breaches can have, seriously threatening the financial health of the organization. CFOs are also beginning to align their oversight of risk and compliance with the proliferation of cyber threats, providing an added imperative for CFO involvement in cybersecurity strategies.

A broad range of organizational information underlies everything that the CFO and his team do on a daily basis, and much of that information is both highly sensitive and crucial to the business strategies of the organization. The CFO must have constant knowledge of where this information resides, who is working with it, and the plans to keep it secure. And now CFOs must have heightened awareness of where cyber threats are coming from and the many ways that intruders can get to that information.

Besides working to ensure the safety of corporate information, CFOs also must assess the risk and potential impact to the organization should a hacker successfully breach security measures. The damage to an organization is not just determined by the direct cost of the cyber attack. As many well-known corporations have demonstrated all too often, there are often long-term negative consequences related to the loss of trust and a tainted brand reputation.

Investments in cybersecurity technology solutions and services are increasing each year. Managed services providers are finding that their clients want and need a partner who combines knowledge and expertise for cybersecurity itself with the implementation of effective technologies that can keep pace with fast-changing methods of attack. And MSPs must understand the responsibilities of a corporate office like the CFO and the mandate for secure corporate financial operations. For MSPs to do well, understanding business objectives, processes and corporate behavior is every bit as important as top-notch technology skills.

Every new technology development translates into new ways for hackers to digitally threaten organizations. The in-depth and broad range of MSP expertise for many technologies and business processes is invaluable for helping CFOs not only grasp the current state of security in their organizations, but better visualize what the future may hold for new kinds of cyber attacks. As organizations adopt new technologies, they should also develop new plans for protecting digital corporate assets.

The future of managed services depends on these new ways of working with an organization, to best support the goals and business outcomes that the organization seeks. MSPs can no longer simply look to traditional IT roles and ways of working. Sophisticated technology and the digital era demand that managed services and solutions always look at the bigger picture and work with more non-IT roles. Innovation calls for new approaches to help organizations attain success.

 

Image source: IBM

 

 

This post was brought to you by IBM for MSPs and opinions are my own. To read more on this topic, visit IBM's PivotPoint. Dedicated to providing valuable insight from industry thought leaders, PivotPoint offers expertise to help you develop, differentiate and scale your business.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting

 

 

Connecting the Cyber-Threat Dots through Big Data

 

The managed security services market has been in play for more than a decade. Not surprisingly, it continues to show vibrant growth, fueled in part by cloud-related factors. Research and Markets, in a January 2015 report, estimated that market growth will run from $14.3 billion in 2014 to $31.9 billion by 2019 (with a CAGR of 17.3%). Growth for security services touches just about every industry, and all sizes of organizations.

Managed services providers are tasked with marrying expertise in cybersecurity and cloud with big data analytics to deliver comprehensive protection and proactive measures to their clients. Proactive cybersecurity services call for the implementation of far-reaching and sophisticated data and event management technologies and services, to improve the aggregation and analysis of security event big data. Managed security services should address the imperative for integrating many disparate sources, including big data sources, to derive event correlation that can be aligned with other analytics to continuously improve alerts, preemptive processes, and real-time reporting.

Today's cyber security threats frequently operate as patterns that usually deviate from the expected behavior for most authorized users or for activity on particular devices or IP addresses. To identify and fight such attacks, security teams need technology services that can find and analyze deviant trends. This is a natural fit for big data mining and analytics combined with state-of-the-art cybersecurity programs.

Machine-generated data provides fertile ground for using big data analytics to root out cyber security threats. Machine data lives in the IT infrastructure: network logs, event logs, firewall and security system data, web logs, email logs – anything and everything operating in the infrastructure. But machine-generated data can be quite problematic for aggregation, data mining and analytics. Such data must be processed very quickly, frequently in real-time and usually exists in large volumes that are continuously proliferating. Machine data sources are quite variable, many of which are multi-structured formats that further challenge data mining efforts.

Once data has been extracted from machine-generated sources it can be enriched with other kinds of data to establish context, and elicit patterns and trends related to cyber attacks. Managed analytics services can take on a forensics quality while searching through data for patterns of irregular or unexpected activity. For instance, realtime endpoint forensic data capture and analysis is on the rise to plug yet another hole in systems infrastructure, in order to block data breach entry points.

Situational awareness is another important aspect of fighting cyber attacks. The overarching value of situational awareness is continuously knowing what is going on around the organization, to "instantly" respond to certain events. Big data analytics strengthen situational awareness primarily through fast real-time assessments that can reduce time to decisions and actions in response to potential threats and certain anomalies. These sorts of analytics also require corroboration from subject matter experts, to help make the best decisions based on the data at hand. And this opens up yet another "cybersecurity frontier" for managed services providers where knowledge and expertise are just as important as optimally running technologies.

 

Image source: platfora.com

 

This post was brought to you by IBM for MSPs and opinions are my own. To read more on this topic, visit IBM's PivotPoint. Dedicated to providing valuable insight from industry thought leaders, PivotPoint offers expertise to help you develop, differentiate and scale your business.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting