Highly Competitive - software industry insights

Always-On Brand Experiences

 

Midsize firms often adopt newer technologies faster and change strategic direction more agilely than larger enterprises. Both of these actions can provide competitive edge over the increasingly commoditized worlds of their larger competitors. Fast and agile strategies are also needed to respond to the ever-changing nature of customers. Customers are more proactive about their purchases, which calls for right-time brand experiences, often over multiple channels. Marketers now must be dynamically responsive to change and must more quickly decide what to do.  

Source: bigthink.com

To provide a unique brand experience for individual customers, companies and their marketers have to target real people and their desires of the moment, not statistics or generalities. Adaptive marketing, a new version of direct marketing in today's digital era, is a promising approach for crafting such individual brand experiences. Companies become customer-focused organizations that are committed to constantly re-crafting product offerings, sales initiatives and marketing tactics to match individual customer needs and wants.

Marketing technologies and processes have to be transformed to meet the fluid requirements of being "always on". Media and agency relationships have to evolve to handle constant change. Digital asset management has new importance as a management piece for customer-responsive images, marketing calls-to-action, and other personalized and time-sensitive content.  

Customer intelligence drives adaptive marketing processes, and ultimately the goal of continuously individualized brand experiences. Right-time analytics are invaluable to all marketing solutions, constantly reflecting changes in customer behavior and market trends. Analytics help unlock the essence of social and customer interactions: synchronizing the organization with customers, their interactions and their communications related to the organization and its products.

Data is now strategic, and data management initiatives must ensure accurate and timely data for fast-moving adaptive marketing. The data and analytics that support adaptive marketing initiatives are also key tools for the midsize enterprise anyway: intelligence to better serve customers, develop the right products, and discover new opportunities and markets. This is where competitive advantage emerges from how well a company tracks and measures customer behavior and interactions that will help them understand and interact with customers on an individual level.

Companies will never "own" the customer or the customer experience, but they can proactively provide and nurture the right brand moments that can attract and influence the customer to make purchase decisions. The best outcome for adaptive marketing is when the company meets customers on their terms, and crafts a positive brand experience every time. The best outcome for the customer beyond quality brand experiences is the value received in every interaction and purchase journey.

 

 

This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet. I've been compensated to contribute to this program, but the opinions expressed in this post are my own and don't necessarily represent IBM's positions, strategies or opinions.

 

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts both software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

                        

 

The Adaptive Enterprise: New Directions for MDM?

Originally published in Hub Designs Magazine, November 2012

 

As market forces continue to fluctuate, business leaders are searching for the best ways to work with uncertainty and still deliver improved performance. As a result, organizations are transforming into adaptive enterprises.

Performance for the adaptive enterprise means agile, dynamic, and responsive to continuous change. But this kind of performance requires enterprise data that is reliable, usable and timely, so that analytics and decision-making processes are quickly on target with relevant recommendations.

Information management has an important new challenge for provisioning right-time data to emerging adaptive enterprises. MDM is a reflection of how an enterprise uses data and information for business purposes. The creation and management of master data touches more than the information itself. By creating data repositories and processes that reflect business functions, enterprises should now have access to the information that is so vital to effectively and agilely adapting to constant change.

 

 

Agile Processes, Agile Data

In 2011, Gartner surveyed 300 user organizations: 49 percent said process improvements are the key business priority driving interest in MDM. Gartner's Andrew White commented:

"MDM treats information as assets that indirectly contribute to enterprise performance by improving process execution via data consistency," White wrote in his presentation slides. "BPM is a management discipline that treats processes as assets that directly contribute to enterprise performance."

MDM has a clear role in improving business processes; both MDM and optimized processes contribute to better performance management. The increasing focus on master data in relation to business processes extends usage across the enterprise to more business roles and functions. For many enterprises, data governance policies are the means for the orchestration of master data management, practices, processes, and people, so that compliance with data governance policies assures that master data is available as a trusted and vital asset for the enterprise. A new direction and challenge for data governance will be creating policies that reflect the continuously changing needs of the adaptive enterprise.

 

Dynamic Business Processes: Adaptive Case Management

Adaptive case management is an interesting example of adaptive and responsive processes that depend on reliable data. Forrester Research defines adaptive case management (ACM) this way (Forrester calls it dynamic case management):

A highly structured but also collaborative, dynamic and information-intensive process that is driven by outside events and requires incremental and progressive responses from the business domain handling the case. Examples of a case folder include a patient record, a lawsuit, an insurance claim, or a contract, and a case folder would include all of the documents, data, collaboration artifacts, policies, rules analytics, and other information needed to process and manage the case.

Immediate access to the right information is essential to ACM due to the unpredictable nature of case management knowledge work. 'Unpredictable' is also a driving force of the adaptive enterprise, where there is increasing need for organizations to adapt to the volatile nature of business as it happens. ACM may evolve beyond its 'roots' in business process management to become a much-needed internal platform of process orchestration, practices and technology to more relevantly and quickly surface the most-needed information, content, and data, wherever they exist in the enterprise -- all in the context of situational business processes related to a particular employee's responsibilities.

Frequent mandates for ACM include managing risks and compliance, containing costs, applying a systems approach to non-conforming processes, and the ability to respond well to the unexpected. MDM parallels these mandates with a number of its value areas for enterprises:

• Compliance and risk management
• Revenue growth
• Customer interactions
• Cost and performance efficiency
• Business agility

Both MDM and ACM can contribute positively to customer-focused activities, along the lines of real-time analytics, decision-making, dynamic process, and timely responsiveness.

 

Performance, Outcomes, Metrics

Obviously, ineffective business processes and data bottlenecks are obstacles to the adaptive enterprise that needs to nimbly change in response to events and new insight, and to rapidly make decisions to make the most of new opportunities and challenges. Achieving performance improvements starts with identifying how data-fueled processes can be optimized, and how MDM and BPM can work together to do so. Well-functioning business processes, whether explicitly defined or adaptive (as in case management), are an essential contributor to agile business performance.

A classic example of setting an improvement objective aligned with MDM is the reconciliation of customer identity across systems of record and systems of engagement, since both domains must be integrated to support and capture customer interactions. Many business functions - and strategies - greatly benefit from a unified view of the customer. The organization can quickly and accurately interact with customers. Now metrics can be applied to measure outcomes and performance relative to customers, based on the organization's KPI preferences, and tied to corporate and line-of-business objectives.

Processes and performance improvement initiatives can be decomposed into master data elements that can be sorted and aggregated properly for metrics to quantify value. Focus is put on the data assets that are the most essential for overall business improvement. MDM makes such a focus repeatable across the enterprise to integrate business processes and metrics around the same master data elements

 

The Successful Adaptive Enterprise

Success factors for enterprises almost always include effective leaders, the right strategies, and high quality products and services that meet customer needs. But more and more demands are pushing enterprises to constantly monitor and respond to changes in markets, customers and competitors. These changes will then alter other success factors: strategies now have a shorter shelf life. Enterprises are challenged to:

• Transform to an organization comfortable with continuous adaptive behavior while relying on business processes and data systems to increase flexibility at tactical and strategic levels
• Make better use of data, processes, analytics to improve decisions, to predict future trends and markets, and to proactively manage change
• Develop responsive risk management and compliance processes that are ready for the uncertainties of business, both good and bad

Data analytics are seeing wider adoption as important tools throughout the enterprise, reflecting the convergence of BI, BPM and MDM. Analytics are also being used to evaluate enterprise data to improve both information management and business processes, more frequently in real time. Analytics-fueled data management capabilities bring improvements and evolution to data integration and data quality approaches. More enterprises are looking for analytics embedded in a wide variety of systems, including operational systems that now make use of real-time analytics and decision-making technology.

MDM initiatives have the right capabilities and strategic purpose to underpin business agility and the adaptive enterprise. Master data can be extended to provide improved use of information systems and repositories throughout the enterprise. MDM can stretch its data muscles to take on new business requirements and objectives where data and process are key components, and now venture into new categories that need MDM to better serve the enterprise. To achieve agile business actions takes highly disciplined approaches to MDM and data governance; and it takes flexibility bred into information management DNA to be able to quickly but reliably handle change.

 

Related Links

Business Priorities Point the Way for MDM

Connecting Data Governance to Business Outcomes That Matter

Ungoverned Spaces: Innovating Information Flow

'Big Data', Intelligence and Multi-faceted Innovation

Data Management: Reaching into the Cloud

Information, Intelligence and Process: Combining Forces to Better Answer Business Needs

The Ever-Expanding Importance of Analytics–True Intelligence for Business

Internal Disruption–No Longer Business As Usual

"Lean Is Not a Destination, It's a Journey": Lessons in Quality, Reliability, and Risk for MDM

 

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts both software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

 

  

Data Governance: "Lean Is Not a Destination, It’s a Journey"

 Lessons in Quality, Reliability, and Risk

 

 Originally published on Hub Designs Magazine May 2012

 

"Lean" and "Agile" concepts have infiltrated aspects of many different kinds of businesses, from managing manufacturing processes to developing software. While there are several principles that define Lean processes, one of the most important ones is to provide more value more quickly to customers (both internal and external). There now seems to be interest in applying Lean and Agile to data governance, including very specific applications for data integration, MDM and data quality.

One of the difficulties for discussions of Lean/Agile concepts is that different constituencies in different industries and professional roles can have disparate definitions for Lean and Agile. Also important to remember: Lean and Agile have resulted in varying results, some good and some not-so-good. The following list covers what many who advocate for Lean consider the core principles (but many variations - and debates - exist):

The five-step thought process for guiding the implementation of lean techniques is easy to remember, but not always easy to achieve:

1.     Specify value from the standpoint of the end customer by product family.

2.     Identify all the steps in the value stream for each product family, eliminating whenever possible those steps that do not create value.

3.     Make the value-creating steps occur in tight sequence so the product will flow smoothly toward the customer.

4.     As flow is introduced, let customers pull value from the next upstream activity.

5.     As value is specified, value streams are identified, wasted steps are removed, and flow and pull are introduced, begin the process again and continue it until a state of perfection is reached in which perfect value is created with no waste.

The fifth principle is the well-known concept of continuous improvement, which underscores initiatives for data governance and management, as well as data quality. If an enterprise is working on establishing an over-arching data strategy along with sensible processes and practices to enact the strategy, the enterprise needs concepts that will lead to very good results. Lean and Agile are tempting roads to travel, but will they take the enterprise to an optimal "destination"?

 

Lean Data Governance

The mantra of Lean management principles is often: Lean Is Not a Destination, It's a Journey. The reality for data management and governance initiatives is that the work is never done simply because data-related processes must evolve as the business and its markets change and contort, and as data strategies mature and point the way for providing more value to business needs.

We know that data governance is a convergence of many areas that directly impact the relevance and usability of data in an organization: besides data-related practices and solutions, business process management and risk management are also part of the mix. Even more important are considerations for people and politics (or 'culture'). Lean data governance must reflect all of these areas, or undesirable results can easily occur. Risk management, as a means to minimize risk levels for operational and other processes, has synergy with Lean principles through concepts such as eliminating "waste", bringing more value to customers (internal and external), and engendering continuous improvement.

 

Lean Data Integration

Recently interest in lean data integration has increased as organizations seek faster means for integration processes, while still effective and comprehensive. Forrester lists "lean integration" as one of the Seven Top Integration Trends For 2011 To 2012. Paralleling the principles of Lean for manufacturing and adding Agile software development approaches, Forrester's take on lean integration includes:

• Delivering quickly. Agile development techniques like Scrum (an iterative, incremental software development methodology) are a key component of Lean Integration, and this approach helps developers deliver results faster than using traditional techniques.
• Making continuous improvements. Integration tools support continuous improvement with features that enable easy modification, manipulation, and configuration of a wide range of application metadata artifacts, such as rules, policies, mappings, and processes. Speeding such change makes it easier and faster for developers to respond to the rapidly changing requirements typical of Agile processes.

Forrester expects many organizations to move aggressively to implement Lean Integration to speed up application delivery, especially those that value rapid delivery because it drives increased revenue and/or customer satisfaction (see Figure 4).

 

 

    

    Source: Forrester, Seven Top Integration Trends For 2011 To 2012

 

 

The Quality Lesson from Toyota Lean -- for Data Governance

For years, Toyota epitomized the promise of Lean manufacturing – after all, Toyota invented Lean Manufacturing. But Toyota painted itself into a corner when quality was compromised and dangerous part failures continued to plague the company for several years:

 

3.8 million cars recalled due to floor mats, of all things, getting caught up in the accelerator.  What makes it so inexcusable is that it is not the first time. 

"Toyota recalled 55,000 Camry and Lexus ES 350 models in 2007 because of complaints of unintended acceleration caused by the mats sticking under the accelerator pedal. The NHTSA said consumers continued to report instances of uncontrolled acceleration in Toyota models after that recall."

So much for quality, and so much for continuous improvement and learning.

 

There is still debate whether Toyota Lean itself failed or if Toyota management failed to act wisely and failed to sustain Lean – likely all of the above contributed to very serious lapses in quality and value-generation at Toyota. Some want to argue that the problems trace back to design errors. But this situation is more complicated simply because of the history of technology enterprises in Japan and at Toyota: Business became Culture. Perhaps the most harmful action by Toyota was to lose sight of the principle of delivering value to the customer, a key component in Lean concepts, and a central business cultural construct.

So what do we learn from this Toyota story that matters for data governance? Quality cannot be sacrificed for efficiency when it comes to data in the enterprise; if the outcome of governance and management solutions is not reliable highly usable data, the solution efforts are useless, and potentially harmful. Lean also parallels the notions of "good enough" data quality and of context setting the rules to decide which data elements must be reliable – and both of these approaches have problematical aspects.

The Lean Iceberg Model was developed to help show the way to a more sustainable use of Lean principles. The idea is to understand the more overt artifacts of useful processes and continuous change -- as well as the more subtle, but very significant, "hidden" artifacts that anchor, enable and sustain Lean processes. And the artifacts beneath the "water" are those of people, practices and politics / culture that are so important to organizational change. The parallels to data governance concepts are interesting and useful.

 

 

© 2008 P. Hines, P. Found, G. Griffiths, & R. Harrison

    Source: "Staying Lean; Thriving, Not Just Surviving" – Peter Hines, Pauline Found, Gary Griffiths, Richard Harrison

 

Like Lean, Data Governance has to be more than technology and processes. Data governance must provide continuous improvement, and potentially faster data improvement results for the enterprise, through the never-ending interleaving of people, practices, processes and technology (and politics…). Healthy and constant collaboration across the enterprise to enable and empower vital data-related processes is a key element for beneficial outcomes and change. Hopefully the growth of cross-team collaboration will also engender faster and better decision-making related to enterprise data strategies.

 

Lean Has Some 'Gotchas'

Going back to the principles of Lean listed at the beginning of this article, the fifth principle presents a potential misstep or flaw that has caused problems for corporations trying to implement Lean and Agile processes, no matter the purpose:

 As value is specified, value streams are identified, wasted steps are removed, and flow and pull are introduced, begin the process again and continue it until a state of perfection is reached in which perfect value is created with no waste.

This fifth principle trips over a highly subjective, vague term: Perfection. Perfection is a chimera, never-attainable, and diverts attention from the realities of evolution, volatility, and change that impact most businesses. The realities of business environments, present and potential, must be recognized as part of the ongoing data governance journey. Organizations should strive for continuous improvement, but perfection is an impossible and useless goal. Competitive enterprises never sit still.

 

 

Related Links:

Business Priorities Point the Way for MDM

Connecting Data Governance to Business Outcomes That Matter

Ungoverned Spaces: Innovating Information Flow

'Big Data', Intelligence and Multi-faceted Innovation

Data Management: Reaching into the Cloud

Information, Intelligence and Process: Combining Forces to Better Answer Business Needs

The Ever-Expanding Importance of Analytics–True Intelligence for Business

Goals and Metrics – Measuring Customer Engagement on the Web

Internal Disruption–No Longer Business As Usual

 

About the author: Julie Hunt is the Editor of Hub Designs Magazine. She is an accomplished software industry analyst, providing strategic market and competitive insights. Her 25+ years as a software professional range from the very technical side to customer-centric work in solutions consulting, sales and marketing. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategic Product & Market Intelligence Services

 

 

Business Priorities Point the Way for MDM

 

Originally published on Hub Designs Magazine March 2012

  

Like data integration processes, MDM solutions can be seen at times as 'plumbing', but in both cases the work to be done has strategic, business-critical purpose. Master Data is meant to be at the heart of most business transactions, applications and decisions that occur in an enterprise. In today's volatile business world, it is important to focus MDM on what matters most for the goals of the business to ensure reliable, unified and timely data for key systems and processes. Such data must conform to how information is used by the business. So while it is tempting for enterprises to think of MDM as a technical or IT initiative, MDM is far more about business purpose and business problems to be solved.  

 

To solve the problem of ensuring high quality 'cleansed' and unified master data, interest in MDM practices and solutions is increasing. But in many ways MDM implementations are still immature and the solution space is still evolving. William McKnight notes that often MDM solutions are focused on "simple" issues, rather than on larger, more strategic value, but that he is "starting to see some pioneer MDM implementations explore the full range of benefits". Among the benefits are data governance practices that elevate data to strategic positioning in the enterprise in support of: operational and productivity improvements, reliability of BI and analytics results, timely customer insights, and ultimately, better decision-making.

 

          
        

  

Understand Business Processes to Understand Data Priorities

First it is imperative to understand how the business operates: how the business uses key data and processes, how processes interact with data, and which business processes matter to strategic focal points. From an understanding of business processes, the best approach or approaches can be determined for implementing a master data solution that not only creates trustworthy, consistent master data but is responsive to changes in the business and to new data that continues to be collected / generated by business processes.

Data quality plays such an important part in the MDM solution. Analysis must be done to determine overall quality, where the weak links are, and the impacts on business processes and operations from inadequate data quality. But data quality is a subjective measurement – part of the process and data assessment will be the setting of requirements for acceptable data quality for individual data elements to provide the level of quality needed by the selected business processes. Some thought should also be given to future needs, whenever possible. The potential "trap" is that data that is "good enough" for one business process may be inadequate for another, particularly when considering future direction.

Data derives its real value from its usability in significant business processes, including those that drive and benefit from analytics, intelligence and decision-making processes. And value attaches to data when it positively impacts business outcomes. Unified, quality data attained through MDM solutions must also be integrated into business processes, which is not an easy task. Such integration is not only at the technology and systems levels, but involves the synchronization of people, practices and 'super' processes across the enterprise to reflect how work is accomplished.

 

          

 

 

Extreme Information Management aka 'Big Data'

All data management solutions are now challenged by the proliferation of data, content and information in sites that are not owned by the enterprise, such as cloud and mobile services, social sites and partner repositories. It has become clear that such data can be valuable to enterprises, particularly for insights into customers, future trends and markets, and directions for innovation. MDM solutions are also challenged to now pull in and harmonize some very unwieldy data with existing master data repositories.

In particular, integrating customer master data with customer-related content in social media will likely lead MDM solutions into new territory. The difficulty of content from social media is capturing the data 'clues' to be passed to analytics for accurately determining sentiment and context. So-called 'big data' needs to be aligned with reliable enterprise master data to properly correlate social content in decision-making processes. How the MDM solution maintains an overall master customer 'profile' will include continuous integration processes for targeted social media sites.

Enterprises continue to experience massive increases in data assets. Beyond social content, enterprises encounter data explosions from business activities such as constant M&A, outsourcing initiatives, and highly diversified partner and supply chain channels. Industries such as Telecom have enormous stores of machine-generated data related to customer activities as well as operational performance and 'product' pricing. GPS data relates more and more to customer and market insights for certain industries.

Achieving consistent master data with such disparate, sometimes unwieldy and complex, data sources will continue to be difficult, but will likely lead MDM practices and solutions to new levels of value to the enterprise and vital business process. Again the focus will have to be on the key business processes that need these new data sources integrated with existing master data.

 

Related Links

Connecting Data Governance to Business Outcomes That Matter

Ungoverned Spaces: Innovating Information Flow

'Big Data', Intelligence and Multi-faceted Innovation

Data Management: Reaching into the Cloud

Information, Intelligence and Process: Combining Forces to Better Answer Business Needs

The Ever-Expanding Importance of Analytics–True Intelligence for Business

Goals and Metrics – Measuring Customer Engagement on the Web

The incongruous worlds of "data integration" – Making data accessible for any-size companies

Custom Code or Data Integration software tools: Strategic Use of Technology for Company Competitiveness

Levels of Social – and Integration – Cut Across Enterprises

 

About the author: Julie Hunt is a software industry solution strategist and analyst, providing market and competitive insights. Her 25+ years as a software professional range from the very technical side to customer-centric work in solutions consulting, sales and marketing. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategic Product & Market Intelligence Services

 

 

Ungoverned Spaces: Innovating Information Flow

 Agile Governance to Ensure that Reliable Information is Available Anywhere Anytime

 

                  Originally published in CMSWire November 28, 2011

 

A good many companies now have Information functions in the Cloud, for a variety of reasons. There is the purposeful selection of cloud services for the usage and management of data, content, records and documents – in other words, Information – that supplements or replaces systems inside the firewall. There is also the proliferation of data and content – more Information – created in many cloud-based sites not under the control of companies. But companies want and need all of this Information -- and also need energetic ways to govern all of this Information as it flows in and out of the spaces used by enterprises.

Going with the notion that for most enterprises knowledge is power, information-led business innovation and competitiveness is becoming a recognized path for potential success. In parallel, services in the cloud are talked about more and more as enablers of innovation, as an agile, dynamic and cost-effective environment for new business direction and experimentation.

With so much information created anywhere-anytime, information governance has a vital purpose: ensuring the flow of reliable information to the right people at the right time. This can also mean reducing former barriers and changing processes, to make more of this information and data are available to more people in the enterprise, sometimes including partners and customers. What should be emphasized is governance that aligns with how information is used in the enterprise, so that governance also connects to business value and corporate strategic goals.

So when we’re talking about innovating information governance, we’re no longer in a vacuum or in a silo, off somewhere in that “information department” – we’re talking about the entire enterprise of integrated business processes and management infrastructure, running on-premises, in the Cloud, and in between. Everyone in the enterprise has to be on board, from top to bottom, across teams, internally and externally, working in concert to enact effective information governance.

  

Priming Information Flow – Anywhere Anytime

There is no doubt that most companies must extend their information strategies to capitalize on highly relevant information that is continuously created in the cloud.  Strong information governance policies and practices must realistically enable the flow of information to continuously bring information to the enterprise -- to improve decision-making, to better understand current and desired customers, improve competitiveness, and support all significant goals and strategies in the enterprise. Information Governance can be a vital part of the ‘framework’ for business agility, responsiveness to competitive pressures, support for new product and services directions, and innovation shepherding.

Enterprises need to break free of old perceptions and implementations of information governance where silos and disconnectedness dominate, and where governance has been viewed as difficult, esoteric, and somebody else’s problem. Companies generally have not taken a holistic view of business information governance, its impacts across end-to-end processes or business units, or the technologies to support it. Instead enterprises have treated information as islands within specific system silos, and have taken the same approach to its governance. Information Governance overwhelms many – many get lost in the details.

If information has become essential to corporate success, then Information Governance must become part of a company’s overall business strategy.  An upper management team, comprised of cross-departmental members, has to set the high level ‘culture’ to enable agile Information Governance:  a corporate ‘infrastructure’ of cross-team collaboration, an enterprise-wide commitment to levels of excellence when connecting with and serving customers, recognition that content from social sites matters, the dynamic means to process all potentially useful data and content no matter the source, and the ability to provide on-demand data and analytics ready for any device.

Individual responsibilities for different components of Information Governance can be segmented to include most groups in the enterprise and to enhance accountability. Services-oriented Information Governance may be an optimal approach with partnerships across teams to enable governance functions and processes. Legal and financial teams could take key responsibility for risk management and compliance, as well as retention policies and practices. IT teams can take on key responsibilities for security, storage, resource provisioning, and user access. It is also now the responsibility of enterprise users of information to understand, partake in and help enforce policies and practices for security, information quality, delegated access, sharing, and so on – these responsibilities are no longer “somebody else’s problem”.

 

    Creating Information Advantage in a Cloudy World

 

Detours For Information Anywhere Anytime

Not surprisingly, persistent issues for Information Governance and Management are mostly the same, whether the repositories are in the cloud, on-premises, or in a hybrid flow based on the two:

 

• No clear holistic strategy for governance of distributed information sources
• Prolific growth of information sources and systems
• Need for extensive integration between information sources, often existing as silos
• Synchronizations to eliminate redundancies and to update with most current information
• Unreliable quality of information, content, data
• Timeliness of information
• Fragmentation that will block information flow and devalue assets

The creation of content and data in cloud repositories has added complexity to ownership and the chain of custody. Enterprises that strongly manage complex chains of information ownership and usage, both internally and externally, will help corporate goals and achieve advantage from all information assets.

Companies must take on the responsibility of monitoring vendors of cloud services to ensure that the vendor is meeting SLA provisions for security, privacy, backup and so on. This monitoring falls both to business users and IT groups. But companies must also develop comprehensive Information Governance plans to supplement what cloud service vendors provide. Companies are liable for proper information governance regardless of where the repositories sit. Business users as well as IT teams must understand the implications of “anywhere anytime” information flows.

 

            IBM: Enabling Multifaceted Innovation

 

Cloud Inspires Governance Innovation

Innovation often relates to finding new ideas and / or applying existing ideas in new ways to bring about beneficial change that affects many aspects of what an enterprise does. Innovation is not limited to the products and services in an enterprise, but also can be applied to business model and strategy, operations, employee and partner enablement.

Again, enterprises need to break out of seeing governance only in terms of risk containment and security (which are important) to also see how it can be a means to innovation. Governance is valuable for setting up policies and practices to guide information usage but it can also provide scope for experimentation and value creation through the orchestrated flow of information to and from many sources. Governance is evolving to handle the challenges of data in the cloud – extending information governance to cloud sources can also help innovate how governance is done. This is a new challenge for teams across the enterprise to work differently and creatively when governing vital information that must constantly flow, while ensuring that the enterprise meets reliability, compliance and risk requirements. Making accountability a distributed function across the enterprise extends the value of information governance and takes it out of its usual silo.

Timely, reliable information in the right hands is more essential as business cycles shorten and become more competitive, and as customer perspective and buy-in become more important. Information created in the cloud along with cloud services comprise a new platform for quickly sharing data and content that matters to the enterprise. “Working the problem” of information governance from the focus of ensuring the open flow of reliable information to be used by many teams in the enterprise anywhere-anytime is a positive move into innovation that will make a difference.

  

Related Links:

Information, Intelligence and Process:  Combining Forces to Better Answer Business Needs

The Ever-Expanding Importance of Analytics–True Intelligence for Business

Goals and Metrics – Measuring Customer Engagement on the Web

Findability Inside The Firewall – Still Trying To Find The Information We Need

The incongruous worlds of “data integration” – Making data accessible for any-size companies

Custom Code or Data Integration software tools: Strategic Use of Technology for Company Competitiveness

Shallow Thinking Ensures Bad Business Decisions

Connecting Data Governance to Business Outcomes That Matter

Data Management: Reaching into the Cloud

 

About the author: Julie Hunt is a software industry solution strategist and analyst, providing market and competitive insights. Her 25+ years as a software professional range from the very technical side to customer-centric work in solutions consulting, sales and marketing.  Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt   For more information: Julie Hunt Consulting – Strategic Product & Market Intelligence Services    

 

 

Connecting Data Governance to Business Outcomes That Matter

Originally published on Hub Designs Magazine September 2011

 

Process-centric initiatives are often optimal approaches for the things that companies want to accomplish. Business processes make it possible to directly connect work to business outcomes that lend themselves to various metrics that provide clarity for business performance. On the other hand, data and data-oriented solutions, by themselves, don’t usually map directly to business outcomes, and as such, usually can’t be measured by value contributed to desired outcomes and achievement of key corporate goals. But -- business processes and data are inextricably bound together. Processes create, modify and consume data – one might even say processes live and breathe data. It is through business processes that data can be mapped to business value.

Companies generally have not taken a holistic view of business information, technologies to improve it, or its impacts across end-to-end processes or businesses, instead treating data as information islands within specific system silos.

Kalido, The State of Data Governance Maturity 2011   

 

Notions like “Big Data” and “Information as an Asset” have stimulated thinking in many enterprises about extracting more value from the data already in enterprise repositories to benefit the business and its goals. But to extract that value from data, it’s not about the data nor is it about data governance. The key focus should be the business processes that impact revenue, business agility, competitiveness, and overall positive performance. However:  both reliable data and effective data governance are key enablers of gaining value from business processes.

Data governance is an important strategy both to maintain data in its most usable condition and to ensure consistency in how processes work effectively across the enterprise, by sustaining the quality, accessibility and relevance of data used by processes. Instead of many silo’d processes working on silo’d data, data governance contributes greatly to connecting essential pieces to synch significant business processes to the bigger enterprise strategic picture.

Martin Atherton on data governance as an enabler of business value:   

For many companies, it is common to focus on the minimum standards required to achieve compliance, instead of seeking to incrementally expand the effort and reap the benefits of being compliant more broadly across the business. In a previous research note, we used the phrase ‘value driven compliance’ to highlight the difference between approaching regulatory compliance as a chore versus seeing it as a business enabler.

 

Some of the difficulty of transitioning how enterprises derive value from business processes (and the data running through them) is the fact that most data-related initiatives have been driven by IT instead of by more business-oriented roles. To enable better business outcomes, data governance needs to bring business roles and IT teams together, as well as incorporating the support of various levels of management and strategic decision-makers. Too many times, data governance has not been implemented strategically. Instead it has been marooned by ad hoc decisions and “policies” tied more to daily operational situations. Without involving data governance in key corporate strategies, enterprises are not in good position to handle internal data issues, let alone address the ever-growing proliferation data from many new sources inside and outside the enterprise.

Ross Dawson expands on the notion of “governance as an enabler”: 

Governance should focus as much on enabling innovation and taking useful risks as about managing and mitigating risk...What needs to happen is that governance, risk, and compliance become part of the process of creating business value...More generally, I think it is important to note that IT governance and corporate governance have long been treated as separate. As I pointed out in my keynote this morning, technology – including in data management – is increasingly central to corporate strategy.

 

With good commitment from management, business units and IT teams alike, data governance initiatives are a very good way to manage data more effectively for greatly improved use in business processes. Data governance should provide the means to coordinate efficient and non-redundant business processes, along with the necessary data assets, across the enterprise, between lines of business. From data governance coordination, standards for reliable data and optimized business processes can be implemented, maintained and changed as needed. Data governance is also an enabler of acquiring important insights into both data and business processes.

 In my interactions with Forrester clients, I get the sense that data governance is receiving the most senior-management-level attention today than I've seen throughout my 18+ year data management career.

Rob Karel, Forrester

 

Tangible Value

Data alone carries no value. Unusable and poorly managed data carries no value. Real cost savings obviously result when data sources are streamlined and cleaned up, when processes are more responsive to business needs. Great value can result when business intelligence and other processes are based on the right data that can be trusted to lead to decisions that are more accurate and timely. The tangible benefits and value of data governance and data are further revealed through their connections to business processes and the desired outcomes from those processes that connect to corporate strategies and goals.

  

Related Links

Data Management: Reaching into the Cloud

Information, Intelligence and Process:  Combining Forces to Better Answer Business Needs

The Ever-Expanding Importance of Analytics–True Intelligence for Business

Goals and Metrics – Measuring Customer Engagement on the Web

The incongruous worlds of “data integration” – Making data accessible for any-size companies

Custom Code or Data Integration software tools: Strategic Use of Technology for Company Competitiveness

Levels of Social – and Integration – Cut Across Enterprises 

 

About the author: Julie Hunt is a software industry solution strategist and analyst, providing market and competitive insights. Her 25+ years as a software professional range from the very technical side to customer-centric work in solutions consulting, sales and marketing.  Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt   For more information: Julie Hunt Consulting – Strategic Product & Market Intelligence Services   

 

 

 

Data Management: Reaching into the Cloud

Originally published on Hub Designs Magazine August 2011

 

In a form of “shadow IT”, Line-of-Business groups have been turning to cloud-based services to quickly set up technology solutions that support their business needs and objectives. IT teams are already carrying heavy workloads with ever-shrinking staffing levels, and frequently don’t have the resources to immediately respond to time-sensitive LOB needs. However without IT involvement, these LOB groups do not usually have the expertise to understand the implications of creating data in the cloud and its relationship not only to data in other SaaS / cloud offerings, but to on-premises systems as well. These business users need a partnership with IT to ensure that comprehensive data management processes are put into practice.

It’s clear that enterprises will continue to increase usage of cloud and SaaS offerings to find new ways to operate more competitively and efficiently. Unfortunately the rapid creation and adoption of cloud and SaaS services have led to increased data silos in the cloud. The same challenges that enterprises face for on-premises data management obviously apply to data repositories in the cloud: data quality and reliability, integration, governance, security and compliance, MDM, as well as improved data accessibility. There is now a bigger picture for data management, where all of these domains need governance as an interoperating whole.

If enterprises fail to address a rigorous data management strategy for cloud computing initiatives, deterioration in data quality and reliability of cloud data are likely to kick in quickly. Data that cannot be trusted nor aligned with enterprise datasets will destroy the value and cost savings that enterprises want from cloud services. One of the most important requirements of business data generated through cloud services is security for that data. Not all data created in the cloud requires security, but most of it does. Comprehensive data management policies are needed in the cloud as well as for on-premises data repositories to ensure secure data processes.

The concerns of both business users and IT regarding data in the cloud have been articulated in many surveys including one recently completed by IDC where issues that relate directly to data governance and MDM initiatives covered security, data integration, operational costs and compliance – all pretty much to be expected when considering management strategies for cloud data. The IDC study goes on to discuss an important point that again requires IT and LOB groups working together: Business Relevance.

The next two challenges – the perceived difficulty of cloud services integration, and the limited ability to customize – are both related to the important issue of business relevance.  While customers certainly enjoy the economic and operational benefits of the off-the-shelf, standardized nature of many cloud services, this survey shows they nonetheless want greater ability to “fit” cloud services more tightly into the context of their specific business.

MDM and data governance provide clear value when they reflect the actual activities and processes of the enterprise. This approach also makes it easier for corporate decision-makers to understand the role MDM and data governance play to support strategic goals and even how they might map (at the high level) to key business processes. Malcolm Chisholm and Fabio Corzo flesh out these ideas in The True Role of MDM: The importance of master data is not just data values or quality, but the real-life concepts they represent… We are looking for fundamental business transformation that will take the business into new ways of perceiving, understanding and developing customers, employees, suppliers, assets and securities. We are looking for business transformation that moves in line with the business goals. And though executives understand that data is at the heart of proper and well-executed business transformation, experience tells us that more often than not, the business transformation is limited to improved data quality side effects that fail to realize the true value of an MDM program. The above concepts become even more imperative when establishing management of data-driven business processes between the cloud and on-premises systems. There will need to be an evolution of data management and governance approaches to deal with new challenges that come from data in the cloud and from an integrated approach to managing data across disparate data domains, many of which are not owned by the enterprise. Cloud services have been very effective in converting (or commoditizing) IT practices through well-formed repeatable processes. To accommodate proper data management for cloud and on-premises solutions, IT and business users will need to work together to build out data management solutions through practices and technologies that support the creation, use and storage of data “anywhere anytime”. Both IT and Business are important to setting the right strategies, including the crafting of a data management architecture that will kick in before an enterprise’s cloud data becomes too distributed for thorough governance. While more and more data will be owned by non-enterprise entities in the cloud, the ownership of comprehensive data management processes and practices continues to fall squarely on the enterprise through business users and IT working to ensure reliable data availability.     Related Links The incongruous worlds of “data integration” – Making data accessible for any-size companies Custom Code or Data Integration software tools: Strategic Use of Technology for Company Competitiveness Levels of Social – and Integration – Cut Across Enterprises  Information, Intelligence and Process:  Combining Forces to Better Answer Business Needs The Ever-Expanding Importance of Analytics–True Intelligence for Business Goals and Metrics – Measuring Customer Engagement on the Web     About the author: Julie Hunt is a software industry analyst, providing strategic market and competitive insights for vendors and buyers. Her 25+ years as a software professional range from the very technical side to customer-centric work in solutions consulting, sales and marketing.  Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt   For more information: Julie Hunt Consulting – Strategic Product & Market Intelligence Services

Information, Intelligence and Process: Combining Forces to Better Answer Business Needs

Originally published as a 4-part series on Hub Designs Magazine May 2011

Data governance can be seen as formalized policies, practices and processes set up to manage voluminous data assets across enterprises. Data governance also sits on an important growing convergence that encompasses multiple, and frequently separate, disciplines: data quality / data integration, master data management (MDM), business process management (BPM), business intelligence (BI) and analytics. The solutions surrounding the management of information, data and intelligence are emerging from artificial silos to acknowledge the greater overlap and interrelationships of these technologies and practices. Deliberate initiatives to tighten up the convergence of these solution spaces will not only improve the better overall functioning of all of these solutions but will help both IT and Business users see how it all works together. A greatly beneficial result should be the elimination of duplicate demands on business users and IT for implementing and managing these systems.

Forrester’s Rob Karel on aligning Business Process and Data Governance Initiatives:

Data governance is not – and should never have been – about the data.

High-quality and trustworthy data sitting in some repository somewhere does not in fact increase revenue, reduce risk, improve operational efficiencies, or strategically differentiate any organization from its competitors. It’s only when this trusted data can be delivered and consumed within the most critical business processes and decisions that run your business that these business outcomes can become reality.

So what is data governance all about? It’s all about business process, of course.

Let’s add this note to Karel’s assessment: business process is about optimizing the Business.

The current state of the increasing interrelationships between Data Governance, MDM, BPM, BI / Analytics (as well as DQ, DI) is also the story of the teaming up of Business users and IT as collaborative partners in doing the work that help businesses meet strategic goals, bring value to company customers, add to competitiveness. Both sides of the story must start with the business problems and the jobs to be done, plus the desired outcomes and benefits. Then decisions must be made regarding the role to be played by the software solutions to be implemented. Starting with the business also encourages the exploration and creation of the specific business cases that reflect needs and requirements. With such business cases defined, monitoring outcomes of data governance, BPM, BI, etc., should have clear basis.

While business processes are a major focus for strategies around data governance, BI and BPM, data still retains a significant role as a strategic asset that enterprises can use to differentiate themselves. These two entities (process and data) must be accorded respect when developing interrelated approaches that incorporate all of these solutions. Just as these solutions are best in synch with one another, so must the enterprise itself work with these solutions across departments and diverse teams. For business and intelligence processes to yield useful, timely and accurate results, the underlying data must be trustworthy.

Knocking down the Silos

Data, information and content permeate most enterprises where business users must become much more directly involved in creating the overall solution for handling these vital resources. Business users usually hold the key to the context for data and processes, as well as future usage considerations. The integrated components of MDM / data governance, BI / analytics, and BPM need to deliver real results to business users but are also highly dependent on participation of business users to reflect reality and provide on-target solutions. True collaboration between Business and IT teams starts with requirements, results, procedures, and then leads to collaborative work on joint projects. Such collaboration fosters reuse, reduces duplication of effort and processes, and builds a network of Business and IT staff that is aligned with how systems support key business objectives, initiatives and future direction.

Commonality also crosses the business drivers for data governance, MDM, BI, and BPM where key benefit areas consist of growing revenue, cost efficiency, different levels of agility, and compliance. Efforts should provide improvements for:

• Faster, more accurate decision-making - real-time intelligence
• Identifying and engendering new markets, new products, new customers
• Enhancing and supporting customer relationships / customer service
• Operational procedures
• Customer cross-selling / up-selling
• Ease of compliance, audits, risk management

Rajan Chandras, Data Gaps Plague Process Initiatives:

For one, the genesis of these types of initiatives is often very different. BPM is undertaken when organizations find themselves beset with process shortcomings, and with the primary purpose of better understanding, improving and integrating business processes. Data governance initiatives, on the other hand, are usually justified by shortcomings in data quality, consistency and integrity.

Then, data governance is often closely intertwined with master data management, and thus organized alongside important "master" business data entities -- customer, product, organization and such.

That said, Rob Karel, Forrester analyst and lead author of the report, suggests that IT must learn how to better educate and evangelize data issues in a language and a context that matters to the business -- a responsibility that Karel puts squarely on IT, "because IT often has a more cross-enterprise view than siloed business units and functions."

For data governance and process governance efforts to be successful, they both (business and IT) must frame their priorities and business value in the context of which business processes they are aiming to improve, transform and optimize, says Karel.

From Forrester study, September 2010

Source:http://alteryx.com/How-Alteryx-Performs/Pages/White-Papers/Forrester-Report-on-BI-Platform-Requirements.aspx

Intelligence, People and Process

MDM is a reflection of how an enterprise uses data, information – and content - for business purposes. The creation and management of master data touches more than the information itself. By creating data repositories and processes that reflect business functions, enterprises should develop access to the information that is so necessary to effectively and efficiently achieve business goals. It’s extremely important to analyze, manage and provide access to all forms of information – structured and semi-structured data, as well as “unstructured” content.

For BI and analytics solutions to provide real intelligence, they must be based on accurate and timely data sources. BI outputs are based on data aggregations, until recently, from structured data in data warehouses, with delivery in report or dashboard form to enterprise users. Analytics add the dimension of mathematics and formulas, with variants addressing forecasting and prediction.

Neil Raden responds to the ebizQ question: How does master data management change BI?:

…to describe how organizations collect data as part of a process, but manage to make it infinitely more valuable by using it for other purposes.

What does this have to do with BI and MDM? In our research, we have found that most knowledge workers shun BI for two reasons (not performance or ease-of-use): relevance and understanding. MDM adds nothing to address these concerns because its representational framework, a relational schema, is inadequate. It's a representation of a model. An ontology is a model. Until MDM rejects the relational model as its underlying schema, it will be unable to add the rich meaning, relationships and even reasoning that an ontology can do.

So, the point is, if you're going to go to the massive effort and expense of an MDM solution, take some advice from 21 years ago…Make the data useful for people, not just governors and black belts.

One newer area of interest for BI / analytics solutions is the inclusion of collaborative activities to add contextual and qualitative layers to the output of BI processes. To achieve authentic intelligence, contextual / qualitative layers can provide a strong basis to test, fine tune and filter the artifacts of analytics. Analytics can benefit greatly from human filters that bring experience, knowledge, creative thinking. Context has a big role here: context for sources, context for outcomes, context for usage with other data points, to achieve the optimal Intelligence for “making better business decisions”.

The possibilities for new applications of analytics increase with collaboration. Inviting in many-to-many interactions also opens up processes to new ideas from participants. Gartner found that social venues and collaboration help to track and capture outcomes of the decisions made based on BI / analytics:

Gartner's user surveys show that improved decision making is the key driver of BI purchases. However, most BI deployments emphasize information delivery and analysis to support fact-based decision making, but fail to link BI content with the decision itself, the decision outcome, or with the related collaboration and other decision inputs. This makes it impossible to capture decision-making best practices. Solutions are emerging that tie BI with social software and collaborative tools for higher-quality, more transparent decisions that will increase the value derived from BI applications.

With convergence, employees in the enterprise should operate more effectively, where improved data governance/MDM lead to better BI, where collaborative processes also enhance and validate BI, where a collaboration setting for BPM works to deliver more of the information that the enterprise needs. A possible approach to marrying collaboration, data and intelligence to business processes, and, more importantly, to the way people work, can be seen in Tibco’s re-work of tibbr. Dennis Howlett provides this description of tibbr:

It intelligently marries people, process and context, delivering information the way people want to consume

Tibco connects tibbr to business processes and event-triggering that are then exposed in tibbr for taking action. While tibbr is built more for real-time information streams than archives of content and information, tibbr and Tibco have created a platform with a lot of potential for improving overall information findability that adheres to context and worker roles. It connects to process, people/workers, collaboration venues, data and information streams, centralizing all event streams into one dashboard. Information can be organized by subject or topic, rather than by people. tibbr enables users to create, contribute to, and subscribe to the real-time event streams that matter most to them.

Convergence Clarifies Benefits and Value

In a recent Gartner report covering predictions for data integration and MDM:

Through 2015, 66% of organizations that initiate an MDM program will struggle to demonstrate the business value of it.

It’s probably not a stretch to extend this “struggle for value” to BPM and BI / analytics. But through the convergence of MDM with BI / analytics and BPM, an analyst like Sandy Kemsley sees increased business value, in this Loraine Lawson interview:

Lawson: It's become more important with MDM to identify pain points and a real business case. Yet MDM is still pretty IT-centered. Does connecting it to BPM allow it to be more business-focused as a project?

Kemsley: Absolutely – connecting it to BPM, and also connecting it to business intelligence and analytics, because that's one of the things that many business users are interacting directly with analytics, whether it's just simple reporting or dashboards or a more complex, drill-down. You can get better analytics if you have consistent data models and those consistent data models are going to be helped by MDM, that's one of the key business benefits. When you want to have a report that gathers data from a number of different systems, it's not going to be some huge, big, long project because somebody has to figure out what data maps onto what other data. It's going to be a much more straightforward thing because while the data may not all be in one place, at least it will be in a consistent format so that you can easily bring it together.

 

Individually these solutions can achieve some value and success for an enterprise, but at the cost of much duplicated effort and resources. In convergence, with initiatives for business improvement and better decision-making, an integration of BI/analytics, BPM, data governance and MDM has definite synergistic outcomes that can greatly increase business value and reduce wasted effort. Integrating business insights into process execution will significantly improve the quality of decisions and lead to continuous process optimization. And process events kicking off BI and analytics for real-time decision-making increases beneficial outcomes.

All of these solutions should be about solving frequently complex business problems and supporting the work to be done in the enterprise. Each solution brings a significant piece of the “big picture”: better decisions for the business with reliable data feeding both intelligence and process. Ultimately the biggest payoff is how this convergence feeds directly into business agility and continuous change to constantly fine tune sustainable competitiveness.

Related Links

The Ever-Expanding Importance of Analytics–True Intelligence for Business

Goals and Metrics – Measuring Customer Engagement on the Web

I have a confession to make: I am a Hybrid

Findability Inside The Firewall – Still Trying To Find The Information We Need

About the author: Julie Hunt is an accomplished software industry analyst, providing strategic market and competitive insights. Her 20+ years as a software professional range from the very technical side to customer-centric work in solutions consulting, sales and marketing. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategic Product & Market Intelligence Services

 

The Ever-Expanding Importance of Analytics – True Intelligence for Business

“Intelligence” is a word much bandied about in enterprises. Frankly few enterprises use it properly to denote what constitutes authentic intelligence.

"Intelligence is nothing if not an institutionalized black market in perishable commodities"

Master spy novelist John LeCarré

While the above notion is intended to refer to information gathered by spies regarding enemies, this sort of thinking has colored how enterprises view intelligence: the secret information that will solve all problems and kill all enemies. It doesn’t matter if you’re talking business intelligence or market / competitive intelligence. The apparent allure of the military and spy industries for enterprises has badly skewed how many companies view intelligence.

The kind of intelligence that truly benefits enterprises derives from classic definitions of the word: learning, understanding, applying knowledge and experience to new situations to make better decisions. For both business intelligence and market/competitive intelligence, the frequent catchphrase is “making better business decisions”. Both involve data, analysis, recommendations. Both can have strategic and tactical applications. Both benefit from qualitative as well as quantitative dimensions. And both should strive for authentic INTELLIGENCE, not just resultant collections of information.

Business Intelligence methodologies and tech solutions have been used to generate actionable views of a company, frequently rooted in business operations. Until recently, BI outputs have been primarily reports – some might say too many non-contextual reports, many of which may be trivial, pointless, or worse, misleading. The BI world includes analytics, data mining, text mining and predictive analytics.

And sometimes BI is considered to be ‘competitive intelligence’, hence my references to market intelligence. Market/competitive intelligence and business intelligence should be complementary, but are not the same thing. In my view, a certain amount of BI output contributes ‘intelligent data’ for market intelligence initiatives. Overall, market intelligence gleans from many ‘data’ sources from many disparate sources inside and outside the firewall, both qualitative and quantitative, while filtered by specific strategic contexts. Until recently, BI has mostly derived from practices and technology that address structured data and business processes inside the firewall. BI will now have to include unstructured or content sources in analytics, to add in missing subtleties, context, and insight that cannot be pulled from structured data sources.

Analytics Everywhere

In addition to overt BI-style analytics, many other analytics solutions have been proliferating to help make better business decisions. Web analytics were silo’d for a while as relevant only to website usages. But web analytics now have many forms and have become very sophisticated. Data captured by web analytics is now important to many teams across the enterprise and is now becoming more integrated into other “business intelligence” endeavors.

With fast-growing social media applications, SM analytics are now emerging – again the data generated is essential and must be integrated into the overall intelligence base of the enterprise. Content analytics have become even more important as unstructured content proliferates from many sources.

These seemingly diverse analytics cannot be confined to individual silos in enterprises. “Intelligence for the Business” that results from all analytics is needed at LOB / department levels, as well as at the über-enterprise level. And mid-market enterprises need such business intelligence capabilities as well. So all kinds of analytic tools need to become more accessible to many roles and organizational segments that have not necessarily been targets of such solutions before.

Unfortunately, keeping pace with the enormous proliferation of data, and even with the various analytical outputs, is overwhelming many enterprises. More integration between what began as silos of analytics must now happen. Enterprises have to get better at the behind-the-scenes / get-your-hands-dirty work that is essential to make realistic and effective use of all the data (structured and unstructured) pouring in from so many sources. The Semantic Web will likely contribute overall to the integration of structured and unstructured data to add to intelligence relevance.

Predictive Analytics – The Bold New LOB Tool

Considered a subset of BI, but also present in other categories of analytics solutions, predictive analytics may become much more in demand than traditional BI solutions, especially with the new directions of innovative predictive analytics solution vendors.

Eric Segal of Information Management on predictive analytics – citing a marketing use case:

Predictive analytics is data mining technology that uses your customer data to build a predictive model specialized for your business. …The real trick is to find the best predictive model.

…a careful combination of predictors perform better customer prediction by considering multiple aspects of your customers and their behaviors. Predictive analytics finds the right way to combine predictors by building a model optimized according to your customer data.

Predictive analytics builds models automatically, but the overall business process to direct and integrate predictive analytics is by no means automatic - it truly needs your marketing expertise.

Predictive analytics can be part of “traditional” BI offerings – it’s also part of the web analytics world. Baynote provides ‘Adaptive Web’ solutions that include analytics, personalization and social search for customer-focused relevance on websites, to help optimize engagement and retention. Baynote’s Collective Intelligence Platform provides companies with predictive analytics based on the implicit patterns of customers visiting their websites. Omniture, WebTrends and other web analytics solutions also provide capabilities for predictive analytics.

Content Analytics, Content Intelligence

Enterprises have been challenged for years on the content intelligence front – it has been very difficult and very messy to extract meaning (and thus retaining context) from unstructured content to be used as “data”. The amount of enterprise content is enormous, with geometric expansion in play. For enterprises that publish content on the web, content analytics also offer the means to better connect with target markets and customers:

"Content analytics" can be seen as business intelligence (BI) for/from content, as text (rather than number) crunching that generates insights to improve business outcomes. The two practices, content analytics and BI, certainly share motivations. If you don't analyze your content/data, you may be missing opportunities and running risks.

For content publishers, analytics drives better targeted content delivery, expanded audiences, and secondary uses and new distribution channels. These outcomes add up to profit. On the flip side, they are matched by reduced risk and cost avoidance given possibilities for more complete, more accurate compliance screening, e-discovery, and storage management.

Analytics also boosts value for users. Semantic search, faceted navigation, and content annotation/enrichment create findability and improve user experience and value for users. They also let users treat content like data. Call the goal "content intelligence," enabled by "smart content."

Seth Grimes on Content Analytics

 

“Intelligence” Hits The Wall

Stephen Few’s graphic for “BI Has Hit the Wall” is quite expressive of where BI – and all analytics - need to head to deliver evolving value to enterprises. Stephen Few is a leading expert in creating effective data visualizations that communicate real intelligence and don’t just function as “eye candy”:

Contained in these early definitions was the seed of an inspiring vision that caused people like me to imagine a better world, but the business intelligence industry has done little to help us achieve the vision of the people who coined the term. When Thornton May was interviewing people for his book “The New Know”, he asked a prominent venture capitalist known for his 360-degree view of the technology industry what he thought of when he heard the phrase business intelligence. His response was “big software, little analysis.”

For information to be useful, we must explore it, analyze it, communicate it, and monitor it, but the BI industry’s attempts to support these activities with few exceptions have been tragically comical. The technology-centric, engineering-oriented perspective and skill set that has allowed the industry to build an information infrastructure is not what’s needed to support data sense-making. To use the data that we’ve amassed, a human-centric, design-oriented perspective and skill set is needed.

New World Focus on BI and Predictive Analytics

Some of the significant changes to analytics solutions relate to how final artifacts are “formatted”. There has been a growing transition from traditional report formats to interactive visualization tools, and to collaborative processes that enhance the final artifacts. More companies want BI to help with current and future needs and goals, rather than measuring the past. And these companies should be inviting in more and more individuals (via collaboration) to help refine the accuracy and contributions of analytics outputs. Newer on-demand BI / analytics solutions are providing “right now” intelligence, though still these are still works-in-progress, for the most part.

A number of articles lately have made the point that BI initiatives have not been highly successful for many customers, while making another important point: that “failure” actually is part of the BI refinement cycle. But for costly enterprise-style BI solutions, “failure” can be quite expensive. Gerry Brown of Bloor comments:

However, there is little doubt that the number of BI users overall is increasing. With low-cost operators such as QlikTech, Tableau, LogiXML, Pentaho and Jaspersoft you get an awful lot of BI software for $25,000. Conversely, most enterprise BI vendors don't traditionally get out of bed for deals of less than $50,000. Low cost (or 'free') BI software encourages trial and experimentation with little risk, and this is what most enterprises prefer. After all, BI project 'failures' are common.

Referring again to the graphic for “BI Has Hit the Wall”, Stephen Few sees that “the traditional BI software vendors and most of the industry’s thought leaders are stuck on the left side of the wall” – while the future of BI will come from more non-traditional BI solutions:

The software vendors that are providing effective data sense-making solutions—those that make it possible to work in the realm of analytics on the right side of the wall—have come from outside the traditional BI marketplace. Vendors like Tableau, TIBCO Spotfire, Panopticon, Advisor Solutions, and SAS tend to either be spin-offs of university research or companies that have ventured into the BI marketplace from a long history of work in statistics.

In a very interesting and thought-provoking Gartner Magic Quadrant for BI platforms, the analysts see significant changes in the solutions sought by enterprises:

…there is significant, if not euphoric, satisfaction with, and accelerated interest in, pure-play BI platforms. This is particularly true for smaller, innovative vendors filling needs left unmet by the larger vendors. To understand this paradox, it is necessary to consider a number of factors that are driving the BI platform buying decision today.

Other vendors to watch for new ways to perform and use BI / analytics include Lyzasoft, Predixion Software and GoodData. Lyzasoft and Predixion recently became partners for cloud to cloud analytics-as-a-service. GoodData provides BI PaaS to help speed up solution deployments.

BI SaaS adoption, while very low today, will grow steadily as maturing BI SaaS solutions are delivered in private and public clouds and in on-premises and off-premises configurations by trusted vendors. …innovative, pure-play vendors offering highly interactive and graphical user interfaces built on alternative in-memory architectures to address their unmet ease-of-use and rapid deployment needs. The perceived benefit is so compelling that business users are making this choice, despite the risk of creating fragmented silos of applications and tools.

Business users in particular showed a growing impatience with the time to deploy and complexity of traditional enterprise tools, which led to a rise in departmental buying of alternatives.

Collaboration and the Human Element

One newer area of interest for BI / analytics solutions is the inclusion of collaborative activities to add contextual and qualitative layers to the output of BI processes. To achieve authentic intelligence, contextual / qualitative layers can provide strong basis to test, fine tune and filter the artifacts of analytics. Analytics can benefit greatly from human filters that bring experience, knowledge, creative thinking. Context has a big role here: context for sources, context for outcomes, context for usage with other data points to achieve the best Intelligence for “making better business decisions”.

Collaboration is far more than distributing and sharing documents. It is interactive, inclusive, cross-enterprise. If collaboration is to be part of analytics, then iterative collaborative processes should be established throughout analytics cycles. Collaboration for analytics means bringing in disparate people to test assumptions, validity of data sources, accuracy and relevance of the outputs. These additional participants should provide a wealth of experience, complementary concepts and other essential data and perspectives. Having to prove the relevance and accuracy of analytics results to sympathetic as well as less sympathetic individuals should strengthen BI / analytic processes and projects.

The possibilities for new applications of analytics increase with collaboration. Inviting in many-to-many interactions also opens up processes to new ideas from participants. Gartner found that social venues and collaboration help to track and capture outcomes of the decisions made based on BI / analytics:

Gartner's user surveys show that improved decision making is the key driver of BI purchases. However, most BI deployments emphasize information delivery and analysis to support fact-based decision making, but fail to link BI content with the decision itself, the decision outcome, or with the related collaboration and other decision inputs. This makes it impossible to capture decision-making best practices. Solutions are emerging that tie BI with social software and collaborative tools for higher-quality, more transparent decisions that will increase the value derived from BI applications.

Vendors embracing collaboration as an essential part of BI / analytics include Lyzasoft, Predixion and Tibco Spotfire. Keep in mind that these vendors are all in early stages of building in collaboration as a significant aspect of their analytics solutions.

Analytics Aren’t Always Fun and Games

While there are very interesting analytics solutions that are “friendly” to business users, it is essential that safeguards are in place to ensure that business users understand what they are doing with analytic models and whether the resulting “intelligence” artifacts are correct, meaningful and useful. Collaboration with others who know the data, understand analytics, visualize the big picture, and so on, is one safeguard to ensure reliable analytics outcomes and correct usages.

In a particular enterprise, do enough people know what to do with analytics, both to start processes in meaningful ways and to audit outcomes to validate accuracy and relevance? Are users chasing the right problems or questions, providing the right data sources, including enough pieces? Do they have the understanding to work in a “big-picture” sense? Yes, analytics vendors should build in methods for validation, testing, guidance, but is that enough? Bob Warfield addresses this concern:

Few enterprises have the right analytical talent.

I was musing not long ago with VC and fellow EI Evangelos Simoudis that very few people actually know how to ask questions in a way that solves problems. It is something of a Sherlock Holmes conundrum. All the data is available. It is shatteringly obvious once someone connects the dots. Yet, very few know how to step across the stones that peek above the raging torrent of data to get to the other side where the answer lies without falling in and getting wet.

“You see Watson, but you do not observe.”

Beyond the traditional users of BI / analytics, there is an additional “challenge” to vendors to reach potential users who “need predictive analytics but don’t know it”. The challenge comes in two parts: first to find them and connect meaningfully with them; and then to provide guidance for every step of using the analytics solution to ensure proper outcomes. To connect with these “new” analytics users, vendors have to help them understand why they need analytics and how to use it. Self-serve training modules must be available for using the solution but also for understanding what analytics mean to the user’s role, industry and business needs. Detailed templates and best practices organized by industry scenarios are also important for success. It will be interesting to see how successful various vendors are for empowering such non-traditional users.

The Elephant in the Room: Reliable Structured Data

Although I’m not addressing data warehousing in this article, I do want to point out the need for basing any data-driven intelligence / analytics processes on reliable data. While it’s the right direction to provide analytics tools that work well for business users and increase ease-of-implementation and ease-of-use, diligence must be exercised to guarantee that data sources fed into analytic processes are trustworthy. Otherwise GIGO.

Teams comprised of IT and business must collaborate to create and maintain excellent processes for data profiling, data cleansing / quality, and data integration. The business users know how source data is used and the context for analytic processes. IT is the partner to help with the data profiling, data quality and integration steps. Data profiling tools can be used by business users to audit data sources, test for errors and/or for the lack of the right data, and to understand different data sources.

"Intelligence is nothing if not an institutionalized black market in perishable commodities"

A lot of data and content that matter for better “Intelligence for the Business” come with a short shelf life of significance. The new direction for many analytics solutions should bring this information more quickly into intelligence processes. When enterprises succeed in achieving richly integrated knowledge and context, authentic INTELLIGENCE is likely to result for “making better business decisions”.

Related Links

Market Intelligence – Making Better Business Decisions at the Core of the Company

Shallow Thinking Ensures Bad Business Decisions

Software companies still fail to tap into top benefits from market & competitive intelligence programs

Levels of Social – and Integration – Cut Across Enterprises

The incongruous worlds of “data integration” – Making data accessible for any-size companies

Custom Code or Data Integration software tools: Strategic Use of Technology for Company Competitiveness

Content Marketing Strategy for B2B Software Vendors: Starring the ‘New’ White Paper

Disclaimer: Julie Hunt is not affiliated with any of the vendors in this article.

About the author: Julie Hunt is an accomplished software industry analyst, providing strategic market and competitive insights. Her 20+ years as a software professional range from the very technical side to customer-centric work in solutions consulting, sales and marketing. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategic Product & Market Intelligence Services

Custom Code or Data Integration software tools: Strategic Use of Technology for Company Competitiveness

 

 

Data Integration software solutions have been in the spotlight these past few years as the explosion of BI and data-driven businesses processes push through companies of all sorts.  It has become clearer how essential trustworthy and timely data is to business, and how important it is for Business users as well as IT teams to participate in decisions regarding data-driven processes, data access, data integration, and overall data quality. Equally important is the collaboration between IT and Business teams as part of the process to select the right data integration software solutions. Business users of applications, and the underlying data, understand the business context and utilization for that data in different systems, and can help decide advantageous ways to manage access and integration strategies.

 

Until very recently, data integration activities belonged almost exclusively to the IT domain. And for some time, data integration has become much more than moving data between disparate systems. While there are many DI software offerings for many different kinds of projects, IT groups have frequently used custom code instead of purchasing DI tools.

 

When focusing on DI software offerings versus custom code, a deeper, important issue for many enterprises comes forward: the need to align IT and Business teams to work on data initiatives from strategic business perspectives, not just from technology infrastructure perspectives. Many line-of-business (LOB) managers are taking on more responsibility for IT projects, setting goals, outlining requirements, providing insight from the business POV, and providing cost justification for projects. These recent changes add momentum to using technology more strategically -- but must involve IT as the key partner in running business processes. Data integration projects can benefit greatly from this sort of dynamic shared ownership.

 

Strategic collaborations between Business and IT for DI initiatives tie directly to the 80/20 IT spend “paradigm”: companies continue to report a problematic metric where 80% of annual IT spend/resources are dedicated to the maintenance of the status quo, while only 20% has been ponied up to drive tech innovations that would likely benefit the competitiveness of the business. With the fast pace of business change and disruption, agility and efficiency in tech utilization are key to building long term success and sustainability. 

 

Transitioning data integration projects from custom code to implementing more DI software tools that provide the majority of the solution - with much less coding – lends significant momentum to help swing IT resource allocation towards innovation rather than maintenance. The reusability aspects and other value points of DI tools add weight to this shift.

 

IT Culture of Custom Code for DI Projects – Tactical status quo?

In a 2008 interview with Philip Russom, TDWI, Lorraine Lawson for IT Business Edge, Russom indicated that at least 50% of DI projects are still hand coded, even though IT managers understand the clear, and frequently significant, savings that usually result from implementing DI software solutions:

“Whenever he explains his case, people tell him: Wow, I get the economics. Wow, thank you for enlightening me. BUT we still have this culture of hand-coding.”

 

Lorraine Lawson followed with additional posts in 2009 – here summarizing the comments generated by the discussion of custom code for data integration projects:

“Last week, I wrote a post on Why Companies Still Hand Code, But Don't Have To, despite the numerous data-integration tools of all price ranges available on the market. In the piece, I summed up the reasons cited by Rick Sherman, founder of Athena IT Solutions, which included ignorance of market costs, a lack of resources and a misplaced adherence to corporate standards.

“Several readers commented on the post, adding that Sherman overlooked one critical reason why so many data-integration projects are hand coded: The human factor.

“When it comes to data integration, the missing human factor is simply that developers like to code. They're comfortable with the tools and, in that moment, it's easier to hand code than evaluate, choose, buy, install, learn and use a new tool – even if that tool might shave months off projects on down the line.”

 

The seeming preference for custom code does point to a weakness in the differentiation of certain DI software offerings:  these tools are still designed to be used as “developer tools”, and indeed involve an amount (sometimes a lot) of coding themselves. Why would developers adopt a DI solution platform that on the surface looks quite a bit like something else that they are already using? Obviously most DI software tools offer a lot more than a “code platform” -- but to have clear differentiation, DI software vendors need to offer more focused solutions that solve specific business problems for specific customer scenarios.  DI tools need to become faster to learn, easier to use, with ready-to-go components, instead of providing general platforms. For better adoption, DI software offerings should be tools that any team will want to use over and over. 

 

The very important functions of data profiling and data quality are significant components of data integration software solutions that cannot be emulated very well by custom code.  DP and DQ processes help derive more value from data and to help render data more trustworthy and usable. Data profiling tools in particular provide a platform for both Business and IT usage and collaboration. DP and DQ capabilities have become important differentiators for DI software offerings, provided they are cost-effective and lean towards ease-of-use.

 

Vendors: Market DI software offerings to be “more findable”

There are DI software tools aplenty for any sort of company initiative – and any sort of company.  They can be on-premise licensed, SaaS subscription, cloud-based metering, open source.  They may manifest as platforms, appliances, mashups, dashboards, APIs, templates, pre-packaged industry-specific solutions, and so on.

 

Since major analyst publications primarily cover only a select group of high-end DI / ETL tools, the notion that companies and their IT and Business teams are not aware of the many options available has validity. However this is really an opportunity for DI software vendors to do more to make their solutions much more “findable” for their target markets and customers. With the upsurge of interactive marketing and social media for B2B selling, “findability” provides software vendors methods to improve the ability of the right customers to connect with the vendor.

 

Competitive advantage can be gained for vendors who do a better job of marketing and connecting what the tools offer to customer business problems. So it’s not just a matter of being findable as a “brand” or category of DI software, but findable as the right fit for a specific DI / business problem, and for a specific customer segment (a segment that could be Business and/or IT). DI software vendors also own the ability to do a better job of providing good solutions that are ready-to-go, to handle specific industry scenarios. No more one-size fits all type of DI platforms – instead, differentiate from developer code platforms by offering focused tools that solve pre-defined problems, solutions that target specific markets/industries/scenarios.

 

When DI software vendors have done a good job of revealing solutions that matter to their markets while showing that these solutions truly solve important problems, the intrinsic value of those DI software offerings should be a “slap in the face” for targeted Business and IT users. The value should be so apparent that IT will rarely want to hand code DI projects again – at least not the important ones.

 

Clear strategic purpose, distinct solution context, and a customer-centric orientation can take DI software further into helping IT change the usual 80% allocation for the tactical-for-status-quo – on to achieving the goal of a larger chunk of resources dedicated to competitive innovation. For DI projects, connecting IT to Business users of company data empowers everyone to work smarter. Savvy companies want to get more from their IT investments and want to rely on their data assets for competitive advantage. Through collaboration the company benefits: Business users know what they need from data and understand the context for data; IT provides the know-how to implement the DI processes to make it all happen. 

 

 

 

Flipping the 80/20 IT spend paradigm – understanding the TCO of custom code for DI

  

Talend Open Source DI Looks at ROI of Custom Code

Whether you agree with all of the Talend conclusions, this study introduces concrete parameters and data about the real cost of custom-coded DI projects.  One valuable takeaway is that this study shows the need to evaluate custom code on the same basis as any software offerings – and why. I find the Talend data believable, since most IT groups overlook lots of hidden costs related to custom coded projects when conducting comparisons to acquiring DI software tools.  Talend is at least opening the door on points that should be considered when selecting the right DI solution. 

 

Further details can be found in the complete report, The Return on Investment of Open Source Data Integration, 2008.

 

Talend ROI Project Category:  Large project: real-time enterprise data integration

· The "Large" project is a complete (large) enterprise data integration project.

· Twelve different databases, two packaged applications (one ERP and one SCM), an application from a partner (accessed through Web Services), a LDAP directory, and a Software-as-a-service CRM all need to be kept synchronized in real-time.

· This project involves 200 complex data flows.

  

Cost to complete this project using custom code:

 

 

Summary of 3 approaches to completing the “large DI project”

 

 

And finally:

 

 

 

About the author: Julie Hunt is an accomplished market intelligence analyst, providing strategic market and competitive insights for the software industry. Her 20+ years as a software professional range from the very technical side  to customer-centric work in solutions consulting, sales and marketing.  Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt   For more information: Julie Hunt Consulting – Market & Competitive Intelligence Services