Highly Competitive - software industry insights

Business People are Drivers of the New Age of MDM

 

Part 4 of a six part series on the "New Age of Master Data Management"

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Master data management and data governance have always been about business. However, there has been a history of MDM implementations putting too much emphasis on technology, and less time into over-arching business aspects. For successful outcomes and continuous improvement, the ongoing participation of various business roles is an essential element of MDM strategies and activities. It takes 'everybody' to ensure optimal data management processes that bolster important business activities.

Organizations need to make data available to more people, which may mean expanding master data management participation across more of the organization. With additional business personnel involved, new ways to use master data will emerge that will benefit more initiatives, strategies, and decision-making at multiple levels. Data consumers may also exist beyond the organization: partners, business ecosystems, third-party developers, and managed services providers.

Data quality is the cornerstone of producing reliable data. Poor data quality prevents organizations from realizing full value from data and can cause serious problems that derail important business initiatives. Business domain knowledge is a critical part of ensuring data credibility. And, of course, it's business people who own the understanding of how data is used, for which processes, and what future uses may be in the offing.

For all of these activities, business roles may need access to the appropriate capabilities in MDM software solutions -- but will come with different levels of technical proficiency.

 

Data Governance Leads the Way for Business User Inclusion

Like many well-defined MDM initiatives, data governance approaches that focus on business cases often better connect data management processes to necessary uses by different business roles. Data governance teams that are comprised of business and technical personnel can take advantage of an array of real world experience to improve processes and practices for tracking data ownership and governance accountability.

A collaborative framework (supported by software technology) is essential for successful data governance efforts. Such a framework can support both direct and indirect collaboration between business and technical roles, ensuring continuous work on defining and managing policies and practices, expanding and clarifying business rules, and tracking activities by all users to maintain compliance and accountability for information lifecycles.

Data management software that offers data profiling capabilities extends an important tool to business users across the organization. Business users can be included in data quality / data relevance processes through continual use of data profiling, to better examine the accuracy and value of data at any point. This is particularly important since a great deal of data comes with a short shelf life of relevance for business operations and initiatives.

Data governance is also a critical component of self-service approaches to data analytics, data integration and MDM. Self-service technologies are growing particularly to provide business users with access to more of the data that is essential for many activities in an organization. There is real risk associated with such self-service data platforms that must be addressed through centralized IT management and governance, which need to be in place to prevent piecemeal and poorly managed efforts scattered throughout the organization. Cloud-based self-service solutions are well-positioned for governance that maps into organizational data governance practices and policies.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Business User Access and Pervasive Master Data Management

Master data management appears to be on the threshold of becoming a pervasive platform. 'Platform' refers to both a key go-to initiative for data-smart organizations and to the integrated technologies that underpin master data management. Making MDM pervasive comes from a technology solution base that equips organizations to handle any data-driven scenario and support any business case, where business and technical roles take on innumerable data-related tasks. It's a mindset of business and technology teams working together to use data to improve business results and competitive edge. This is aided by increased access, as appropriate, for less technical business users and for line-of-business developers who don't have deep MDM expertise.

More MDM software vendors are recognizing that different user roles, especially business roles, comprise a major sustaining driver for the future of master data management. When vendors create role-based UIs and capabilities focused on less-technical users, the work they do also benefits the UIs and capabilities for the more technically adept. Organizations benefit greatly from better software design and the push to expand what MDM technologies can accomplish for diverse MDM activities, and for the people involved in completing them.

Of course, it takes very sophisticated technologies behind the scenes to deliver "simple" UIs that give everyday business users access to MDM and data governance capabilities. A lot of work must be done especially to build in safeguards to keep less technical business users out of trouble. To ensure that everyday business users do the work correctly and avoid serious missteps, each category of MDM activity has to be specifically defined, with straightforward guidance at every step.

If an MDM solution doesn't provide pre-built objects, IT teams should take on the development of appropriate (and reusable) workflows, and other objects to enable business user participation, while building in safeguards. IT teams also have the responsibility to develop guidelines for business users, and then monitor and assist these users as needed.

Well-structured MDM initiatives can actually move certain tasks, like exploratory analysis, from technical teams to business roles. Business domain experts can take on responsibility for managing and validating business rules and data hierarchies that operate optimally for business needs. The right business roles can also create, validate and enhance approval workflows.

 

Drivers of New Age MDM

Tremendous potential exists for the increased participation of business people in MDM and data governance activities. Opportunities are opening up for business users to utilize powerful MDM software capabilities to explore, manipulate and access data without constant IT involvement. There is obvious advantage to organizations that fully support and empower business users to work more directly in many aspects of data management. When technical and business teams cooperate to engender improved collaboration, many MDM activities can be performed quickly and effectively, while ensuring that data governance practices and policies are upheld.

Organizations would be wise to develop management plans to assure that business users are empowered but also protected when working with MDM platforms. Cooperative action is also needed from technical and business teams to ensure that business user "protection" doesn't inhibit necessary access and use of capabilities.

Executives in many organizations understand the value of trustworthy and relevant data for improved operations, productivity, competitiveness, and overall decision-making. But these executives still struggle with creating the strategies and organizational transformations that put usable data in the hands of many more business users. With data as a clear strategic must-have, organizations are obligated to do more to build out a comprehensive plan for business user participation and then actively work with teams to execute it.

 

Image source: Native American Weaving

 

About the author: Julie is an independent consultant and industry analyst for B2B software solutions, providing services to vendors to improve strategies for customers and target markets, products and solutions, and future direction. Julie has expertise in several solution spaces including: data integration and data quality; analytics and BI; business process, workflow and collaboration; digital marketing, WCM and social media; and the pivotal importance of user and customer experiences. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

Data for Everyone? Self-service Data Integration

Part 4 of a multi-part series on the essential interwoven nature of Business and Modern Data Integration

 

                                                   

 

The notion of self-service data integration has been around for a while, somewhat hand-in-hand with self-service BI – both primarily coming from the perspective of power business users like data and business analysts. But a door also opened for less-technical LOB business users to participate in data integration activities. SaaS and cloud services introduced self-service data integration where SaaS providers or third-party vendors supply wizard-like UIs for basic pre-defined integrations or data synchs between specific applications. This version of self-service DI gained impetus from the Salesforce ecosystem of data integration services, many of which targeted "everyday" business users.

Data Integration as a Service seemed to be a new force for the democratization of data. However, when looking at current modern data integration solutions, it appears that tech-savvy business users are the main focus of self-service DI capabilities. So what has happened to data integration for everyday business users?

 

 

 

 

 

 

 

 

 

 

 

 

 

Data for Everyone?

The democratization of data has a place in Modern Data Integration, as both conceptual and actual future directions for solutions. Such democratization is meant in the context of sharing data more widely throughout the organization, connecting many more people to the data they need (not just upper level executives). Organizations thrive and sustain success with better and faster availability of the right data at the right time.

Democratization of data includes self-service data integration and other data services, such as data preparation, that can give business users more hands-on access to data, frequently without the direct intervention of IT. But it would seem that there is quite a ways to go to actually connect different kinds of business roles to the appropriate self-service data integration tools. There has been way more talk than actual availability of such solutions.

Of course, it will take very sophisticated technology behind the scenes to keep it simple in the UIs that give everyday business users access to data integration capabilities. A lot of work must be done, especially to build in the safeguards to keep less technical business users out of trouble. To ensure that everyday business users do the work correctly and avoid serious missteps, self-service data integration has to be narrowly defined, with guidance at every step.

Cloud data integration and data prep solutions support dynamic data catalogs or libraries that provide "certified" data sets that can work well for self-service DI. The resulting data sets are available to the appropriate user roles, including the less technical. Governance of which roles access what data furthers the democratization of data in a risk-controlled environment. It's important that business and IT teams continuously validate the contents of such data libraries, as many data sources come with a short shelf life of relevance.

 

 

 

 

 

 

 

 

Data Governance for Everyone

Self-service data integration is clearly moving forward, and it must go hand-in-hand with self-service governance. There is real risk associated with self-service DI that must be addressed through centralized IT management and governance, which need to be in place to prevent piecemeal and poorly managed efforts scattered throughout an organization. Cloud-based self-service data integration solutions are well-positioned for governance that maps into overall organizational data governance practices and policies.

But remember: data governance is more business than tech: business people are essential for implementing successful data governance. Once again, the crucial partnership between business and IT is a core piece of data governance to make sure that it will work well and make sense both for business needs and objectives, and IT management of self-service DI activities.

 

Data Prep Solutions – Blazing a Trail for Self-Service DI for 'Everyone'?

Data preparation vendors have very effectively shared studies that highlight the costly nature of getting data ready for use by organizations: identifying the right sources; cleansing and validating data; testing for context and relevance; creating the appropriate integrations. Business and technology teams alike are better understanding why data integration and data preparation matter, and why they must be done well. These are not trivial undertakings, and require sophisticated technologies to produce the usable data sets greatly needed for many business activities.

An extensive array of data preparation technologies have emerged to help connect more business people to the data they require for analytics and BI, seeking to greatly reduce onerous preparation time to generate that data. Data integration and data prep do have intertwined destinies, since the goal of either should be reliable, business-ready data. Data prep draws on data quality processes that are have been very much part of data integration for quite some time, as has data lineage.

It looks like data prep solutions are creating the next level of tools that more quickly achieve reliable data, by utilizing applications of newer technologies like machine learning. Data prep offerings incorporate focused machine learning / algorithmic capabilities to automate more of data testing and quality as well as approaches to data integration.

Obviously a core aspect of data preparation is self-service through SaaS / cloud platforms. The vendors behind these self-service offerings are clearly interested in extending the variety of roles, both business and technical, that can take advantage of data prep platforms. Right now the primary users are very technical DI experts and tech-savvy business analysts. But various vendors are working to include less-technical LOB business roles, as well as LOB developers. And these vendors do seem to understand that different user roles require different UIs, capabilities, guidance and governance, and access to data.

 

Self-Service Options and Modern Data Integration

We are seeing significant changes in integration processes, where data flows from and to anywhere: on-premises, cloud, partner systems, third-party enrichment, and so on. With Modern Data Integration, application and data integration are converging to provide everything organizations need in a comprehensive platform. Ultimately for business needs it's still all about the data.

And it's about more people in the organization gaining faster access to data to help them do their jobs better, with improved effectiveness and innovation for the business. Through cloud-based platforms, self-service options can be available to all kinds of roles in an organization.

Self-service data integration is an important goal or Modern Data Integration. But self-service data integration - and preparation - still has a long road to travel before data can be accessed by "everyone".

 

Image source: linkedin.com

 

About the author: Julie is an independent consultant and industry analyst for B2B software solutions, providing services to vendors to improve strategies for customers and target markets, products and solutions, and future direction. Julie has expertise in several solution spaces including: data integration and data quality; analytics and BI; business process, workflow and collaboration; digital marketing, WCM and social media; and the pivotal importance of user and customer experiences. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

The Much-Needed Business Facet for Modern Data Integration

 

Part 3 of a multi-part series on the essential interwoven nature of Business and Modern Data Integration

 

Data integration has always faced complex technical issues. Obviously the technically savvy will always be needed for many aspects of data integration, and require solutions that can handle complex problems. But this approach isn't enough for what businesses need and want today from Modern Data Integration. Making room for business ubiquity – business user participation and input – continues to be a challenge.

SaaS and cloud services are important technologies for modern data integration. SaaS in particular has brought into play ease-of-use and focused workflows to help non-technical users move through pre-built integrations. More vendor solutions are improving user experiences and solution efficiency, for both technical and less-technical users. There is increasing development of streamlined UIs, powered by sophisticated technologies behind the scenes, thus speeding data integration tasks.

There seem to be more discussions addressing business data needs, connecting data integration more directly to desired business outcomes via business processes. Including business users in more areas of data management has seen growth both in "lip service" and actual implementations. For the majority of vendors, a business user remains a tech-savvy power user. So we still have miles to go before business ubiquity in data integration includes more types of business roles.

 

 

The Much-Needed Business Facet of Modern Data Integration

Business ubiquity is not limited to direct participation by business users, but includes the overarching notion of Business informing all activities around data integration functions. This means empowering all sorts of business roles to work in partnership with IT, to inform, validate, and provide context to data integration processes. Business roles can help determine if data integration functions will deliver the right data, and will know where problems are likely to occur.

The Modern Data Integration technology platform should provide access points for business users, as well as templates for practices and processes that help organizations engender useful, dynamic and continuous collaborations between business and IT – and the growing involvement of business roles at many levels. Collaborative approaches are usually iterative and involve processes that are both human-based and technology-based. Modern data integration solutions should include all appropriate collaborative aspects.

 

Modern Data Preparation

We're seeing growing attention on tools that more quickly connect business users to the data they need, without much intervention from IT. Data integration is not an easy-peasy undertaking for anyone. But we can break out components that are more amenable to business user access and participation, such as data preparation. There is real value in moving these tasks from IT to business users who can connect data to business processes, usage realities, professional knowledge, and pertinent requirements.

Self-service data preparation solutions include the integration of disparate data sources. These solutions address all of the functions that make the data business-usable and reliable: profiling or exploring, cleansing (which should follow organizational data quality guidelines), enrichment, and so on.

Business users working with data preparation solutions can achieve greater agility to respond to new data sources, new business initiatives, narrow windows of opportunity, impending competitive threats, and unexpected market changes – all of the realities of fast-changing and highly digital business worlds.

 

Business Use of Data Profiling

Data profiling started off as a technology and methodology for IT use. But data profiling can be an important tool for business users to gain full value from data assets. When given the right tools and practices for data profiling, business users should quickly identify inconsistencies and problems for data, before it is used for reporting and intelligence purposes. It's also a sensible way for business users to understand more about the data they utilize in applications, processes and analytics.

From such "data intelligence", business users can have a greater understanding different data sources to be able to ask the right questions for BI and analytics projects. They'll also know if they have the right data and all of the data that they need to answer their questions.

 

Business Data Libraries for Ready-to-go Data

Data libraries are another aspect of improving business-IT collaboration. Modern data integration solutions are doing a better job of attaining the holy grail of business-ready data: directories or libraries of available data views, particularly the data that results from a variety of integrations. This data could be accessed by IT or business users, with the safety net of role-based constraints built in. Such business data libraries greatly aid reusability, as well as allow business users to utilize the data sets that they need, but may not know how to generate.

 

Modern Data Governance for Data and Application Integration

Today's myriad of data sources, data integration tools and approaches, means that data governance is mission critical to ensuring trustworthy and business-usable data. As data and application integration processes overlap more and more in modern data integration, data governance now must oversee all integration approaches, as a centralized function.

Because of the new world of modern data integration, there is more pressure on data governance functions to accept new realities involving business users and self-serve data solutions. Organizations must establish guidelines and processes to directly manage when it's beneficial for business users to perform data prep or integration – or when then work is best done by technical teams due to complex technical requirements.

 

Modern Data Quality

For modern data integration evolution to work well for business user participation, data quality is paramount. More and more organizations understand that data can be a valuable asset, that intelligence can be derived from a variety of data sources, and that analytics of many kinds can greatly benefit decision making, future direction, competitiveness, and innovation. But many organizations aren't well-prepared to tackle the work that must be done to ensure that data is reliable, timely and relevant.

One reason data management and data quality are so important is that a lot of data that can be useful to the enterprise has a very short shelf life. So business processes and activities must be able to tap into data as soon as possible. But that data is only useful if it has high quality. All of the amazing new technologies of modern data integration solutions will mean nothing if the data is unreliable, and therefore unusable.

 

Business Ubiquity – Organizational Symbiosis

Business users increasingly have the potential to utilize powerful capabilities to explore, manipulate and merge new data sources without IT support. There is obvious advantage to organizations to fully support and empower business users to work more directly in many aspects of data management.

Modern data integration solutions should not only support business ubiquity, but can also benefit from it. Vendors can achieve this by making sure that these solutions:

• Provide natural access points for business users backed by built-in guidance to make sure these users don't misstep
• Document and support collaborative processes between IT and business roles in ways that improve data integrations
• Participate in building a comprehensive plan for business user participation and help execute it
• Trace and monitor metrics that are established to connect data integration processes to business outcomes and impact
• "Know" that technology is only part of what is needed to create, implement and reap value from data integration processes
• Support agile change involving both IT and business roles

 

Image source: theyellowsparrow.com

 

About the author: Julie is an independent consultant and industry analyst for B2B software solutions, providing services to vendors to improve strategies for customers and target markets, products and solutions, and future direction. Julie has expertise in several solution spaces including: data integration and data quality; analytics and BI; business process, workflow and collaboration; digital marketing, WCM and social media; and the pivotal importance of user and customer experiences. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

 

Why Modern Data Integration? Core Drivers and Characteristics

 

Part 2 of a multi-part series on the essential interwoven nature of Business and Modern Data Integration

 

From the business perspective, modern data integration has become instrumental simply because a lot of data that can be useful to organizations has a very short shelf life. Business processes, initiatives and other activities must be able to tap into data as soon as possible. Companies need to develop data management strategies that focus on generating business-ready and business-usable data. To do this, organizations need a new mindset where silos of all kinds are broken down, and where business and IT teams continue to grow a collaborative partnership to improve any and all integration processes to more quickly meet dynamic business needs.

For relevance to business needs, modern data integration must ensure business user participation in innumerable data management activities. Line of business teams are the business experts who are vital for revealing key information about data usage, the importance of data to specific business processes, as well as guidelines and thresholds for data quality and remediation. Business users can help to determine the value of source data in context, and to understand how it contributes to intelligence being gathered for the organization.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Drivers of Modern Data Integration

Complex problems confront organizations of all sizes; to solve them, the right technologies must come into play. However, when making technology purchase decisions, more companies now consider the impact on business results. Very few organizations can indulge in "tech for tech's sake".

Until recently, data integration solutions were tools only for tech-savvy teams. With modern data integration, business roles in organizations are taking more proactive participation in making sure that the right technologies are implemented. Vital data and the processes that produce it can better connect to business requirements and objectives. Organizations that recognize - and act on - the strategic value of data, also understand and embrace the need for technologies that get the most out of information and data, with continuous improvement and change.

Another key driver for the emergence of modern data integration is simply the imperative of keeping up with everything data, every day. Data, information and intelligence are the fuel for effective business processes throughout the organization – more of which must be handled in real-time. By considering business processes and business information in alignment to strategies and initiatives, business outcomes are likely to improve.

The need for context also drives modern data integration. The most up-to-date and useful data now comes from many sources, some internal to the enterprise and others from sites all over the digital landscape. For data external to the organization to convey meaning, additional qualification and context must come through connections to data such as master data. Context leads to determining the relevance of data from any source.

 

Core Characteristics

Technology capabilities are part of what defines modern data integration, so it's important to understand key characteristics of a comprehensive platform. Modern data integration characteristics and capabilities should align with both technical and business needs.

• Mandate to keep up with everything data. In this sense, modern data integration technologies, processes and users must be well armed to continuously handle data twists and turns over time. A comprehensive platform is needed to address every business and technical use case related to data assets – and it must be a platform that will continue to evolve and add capabilities.

• A hybrid solution to address any data integration flows. First and foremost the hybrid solution reflects the hybrid infrastructures of most organizations (on-premises and cloud). It also follows both the convergence of data and application integration, and of traditional and cloud-based solutions. This is a further step to eliminate silos for data, business processes and users of data integration.
• A ubiquitous platform: agile, flexible, adaptable, responsive, elastic with universal interoperability, orchestration and connectivity to support any kind of data integration process - anytime, anywhere. Reusability through pre-built components, with categories appropriate for business or technical users.

• Access and support for both business and technical users. Efforts for creating business user access can be applied to simplifying capabilities for technical users. The modern data integration platform should also provide the basis for business-IT collaboration in every sense, to gain expertise from both constituencies for optimal integration processes.
• Services-orientation for reusability. For technical teams, reusable services speed up the creation of more complex data integration processes, and help IT more quickly keep up with business demands. For business users, simpler integrations can quickly be pulled together through wizards.
• Evolved API management. Modern data integration solutions should provide simplified approaches to working with APIs. APIs permeate the digital world (cloud, mobile, web) and the enterprise application world. They drive new ways to access data, to develop apps, and innovate the use of data, including access for third party developer ecosystems, to extend the value of data beyond the organization itself.

• Centralized management for all things data, including data quality, data governance, and master data as critical pieces for modern data integration. These functions also strongly connect to business users, to ensure that the right data in context flows into processes and programs.

 

Modern Data Integration: Differentiating for Business Value

Sophisticated, wide-reaching technologies matter greatly as the basis of a modern data integration platform. But the technology alone does not engender modern data integration. Further definition comes from a solution base that equips organizations to handle any integration scenario and support any business case, where access and capabilities provided to both business and technical constituents take on innumerable data-related tasks. It's a mindset of business and technology teams working together to use data to improve business results and competitive edge.

Ultimately true differentiation centers on the value and benefits delivered to businesses and all users, by taking advantage of modern data integration. Metrics can be put into play to connect modern data integration capabilities and processes to business outcomes; from such metrics certain kinds of business value can be determined. Keep in mind that business value is subjective and different for every organization - and will change over time.

Business should be the heart of modern data integration.

 

 

 

Image source: information-age.com

 

About the author: Julie is an independent consultant and industry analyst for B2B software solutions, providing services to vendors to improve strategies for customers and target markets, products and solutions, and future direction. Julie has expertise in several solution spaces including: data integration and data quality; analytics and BI; business process, workflow and collaboration; digital marketing, WCM and social media; and the pivotal importance of user and customer experiences. Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

 

Data Integration Services Evolution

 

 

With the increased usage of data analytics to help solve business problems and improve strategies, data integration has come more to the forefront as a critical component of overall technologies for data-savvy organizations. New approaches and technologies for data integration are coming into play to better meet what organizations need, especially to support increased business user access to data management itself.

Over the past several years, data integration services have continued to grow and evolve, thanks largely to the cloud. Tapping into SaaS approaches, the early entrants were data integration as a service and purpose-built or pre-built data integrations.

 

Data Integration -- as a Service

Data Integration-as-a-service (IaaS) became possible as more organizations made use of SaaS, cloud services and even B2B data exchanges. Business ecosystems began growing in size and complexity thanks to online connectivity, promoting ease-of-access for B2B relationships and interactions. As more data flowed in the cloud, the need and demand for integration picked up momentum. Originally IaaS functioned as an IT-oriented service, mostly in stand-alone instances. But IaaS instances were soon embedded in other cloud services and applications, as well as cloud infrastructure platforms.

The ability to quickly move data to where it is needed must evolve beyond cloud services that only provide basic APIs and web services that still require users to build connections. While leveraging APIs and web services for loosely-coupled integration processes, IaaS rapidly added more sophisticated capabilities such as virtualized data services, governance, REST-ful services, and agile interoperability at multiple levels.

Cloud-based IaaS has since progressed into Integration Platform as a Service (iPaaS). The data integration services evolution now involves comprehensive platforms that leverage cloud services for the creation and execution of integration processes or flows that connect to systems in the cloud and on premises -- within a single entity or across multiple organizations. Services include critical capabilities: data governance and management, metadata discovery, data quality, transformation rules, agile access to data sources – all delivered with automation and ease-of-deployment as important aspects of the overall platform.

 

Purpose-Built Data Integrations

As a more narrow approach to integration, purpose-built data integration is designed to meet a specific business need, with pre-defined capabilities bundled in the solution. Frequently purpose-built data integration comes as an appliance. Purpose-built data integration services in the cloud comprise an evolutionary step for making data integration services available to business users, to initiate integrations between specific sources or applications.

Cloud services for purpose-built integrations usually include data replication, migration, and synchronizations for customer data. Typically integrations involve specific SaaS applications like Salesforce.com and Netsuite as "pre-set" sources and targets.

These services work with focused use cases where business users do not have to make lots of decisions about how the integrations will be accomplished. Wizards are employed to walk business users through the process. However, at minimum, business users must have knowledge of the sources and targets, and basic understanding of the data integration task, to work confidently and accurately through the purpose-built approach.

 

Business Innovation from Data Integration Services

The evolution of data integration services must align dynamically to the business as it changes. Business users require rapid responsiveness to integration needs, through technology that is 'proactively' ready for many use cases that reflect expected and realistic business situations. Innovations for cloud-based data integration services contribute to optimal business performance and to the faster delivery of valuable data that frequently has a short shelf life of relevance.

Data integration services can tap into the rich value of the cloud and how it connects businesses through common services, with business processes providing contextual frameworks. Such data and process agility underpins must-haves for the competitive organization: always-on intelligence, new business opportunities, innovations in products and business models, robust collaborative processes for the ever-growing business ecosystem.

 

Image source: trendmicro.com

 

This post was brought to you by IBM for MSPs and opinions are my own. To read more on this topic, visit IBM's PivotPoint. Dedicated to providing valuable insight from industry thought leaders, PivotPoint offers expertise to help you develop, differentiate and scale your business.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting

 

Finding Meaning in Big Data Through Metadata

 

 

With the growing interest in big data analytics combined with the shortage of affordable experts and skilled practitioners for deploying and managing big data initiatives, many organizations are looking to managed services as a way to fill the need. More MSPs continue to develop sophisticated services for big data processing, along with all other aspects of data management and analytics.

Big data processing can function well as cloud-based, end-to-end managed services that integrate bi-directionally with on premises data centers and systems. End-to-end services can include streaming data flows, data management activities like data governance, and data visualization and analytics. Big data analytics on cloud infrastructures require top notch management of security and compliance, areas that MSPs are currently improving anyway to proactively move into new technology areas in demand for many companies.

Remember: big data doesn't just refer to high volume or fast-moving data that is important to large corporations. More companies of all kinds need to extract value from a wide variety of data sources – often these sources are difficult and messy for traditional data processing. These sources are also part of the "big data" analytics story and often yield valuable insight. "Messy" data sources include: multi-structured or unstructured data with highly variable formats and semantics like social media content, log files, and e-mail; and machine-generated data from medical devices, industry sensors, automated machinery and systems, and GPS.

 

Metadata Management Services

An important must-have capability for big data processing services is comprehensive metadata management. Metadata provides information about a data item, such as product, that uniquely describes that item. Product metadata can include product ID, product category, supplier ID, size or dimensions, and so on. A field like product ID is also a means for linking to other data sources, for integration purposes. Through metadata descriptors, we can talk about data items in common terms, outside the actual item, and take advantage of metadata to integrate and better understand disparate data sources.

Metadata management is one of the key areas of comprehensive information and data governance. Agile data governance tools are in demand and all sorts of data management vendors are chasing this business. As MSPs expand their presence in data management and analytics services, metadata management and data governance should be of great interest to them. Hopefully the availability of managed services in these areas will make comprehensive data management more affordable and available to organizations of all sizes.

 

What Metadata Does for Big Data Analytics

Metadata is not just about data integration and enterprise data warehouses; for other enterprise needs, metadata helps find data during data discovery, and points the way to interpret and use data correctly. Metadata can greatly streamline and enhance processes to collect, integrate, and analyze big data sources. Working with messy big data like multi-structured sources means that metadata is critical to understanding this data and to connect it to other data sources. Multi-structured data comes with many difficulties – information can be mined, but for it to have meaning and value, attributes like sentiment, purpose and context must be determined as well, and correlated with data sources for customers, products, and so on.

Traditional data sources, such as relational data, provide plenty of logical structure through more easily obtained metadata. However, big data often does not carry a lot of 'native' metadata, so metadata from external sources is essential to unlock meaning. Big data may have to go through certain analytics processes to construct the beginnings of metadata. Then this metadata is correlated with the metadata from other data sources to derive the most useful logical model. As big data processing evolves, new types of metadata may arise to meet the special circumstances of different kinds of big data.

 

Image source: Princeton Tutoring

 

This post was brought to you by IBM for MSPs and opinions are my own. To read more on this topic, visit IBM's PivotPoint. Dedicated to providing valuable insight from industry thought leaders, PivotPoint offers expertise to help you develop, differentiate and scale your business.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

 

Data Profiling Services: Putting Business Data to Work

 

 

Image source: itbusiness.ca

 

Within each business there are many people who possess the knowledge of how data needs to be used for business processes and particularly for how the right data can improve business agility, revenue growth and help meet customer expectations. Such employees are invaluable for examining business data: first, to determine its value in context, and second, to understand how it contributes to intelligence being gathered for the organization. The need to analyze data is at the foundation of every effective data management strategy, whether the analysis is handled from the business perspective or the technology side of the equation.

To connect savvy business people to the data they need requires the right technology tools. One tool that is still not easily made available to business users is data profiling. Data profiling is the process of analyzing data to determine things like its structure and meaning. Users set up a variety of methods and processes to reveal patterns of data usage, metadata matches, value completeness, degree of data quality, and so on.

Unfortunately data profiling has primarily been a tool for IT and data management teams working with data repositories and the systems that produce and consume data. But data profiling should be an important tool for business users to gain full value from data assets. Aided by data profiling, business users can quickly identify inconsistencies and problems in the data, before it is used for intelligence and decision-making purposes. They can also continuously learn more about different data sets to understand how such data fits business needs. Until data is understood in relation to various systems and processes, it's not possible to fully and reliably use the right data for the business.

Here's where managed technology services can put an interesting spin on connecting business users to tools like data profiling that have mostly been the domain of IT teams. Providers can package business user access into managed services for data management, to open up easy usage of data profiling for business purposes. In addition, providers that bring a specific industry focus for such managed services create even greater value for the business.

For example, there are services firms that consult with corporate legal departments and offer managed services packages that blend data management expertise with legal and regulatory expertise. Such managed services include pre-built data profiling services for areas such as risk management and eDiscovery for litigation and compliance. Many organizations are overwhelmed by the complexity and volumes of data that must be monitored and assessed, particularly in light of the challenges of meeting compliance and eDiscovery deadlines. Managed services, backed by industry expertise, can provide a sustainable and scalable solution while reducing the demand on internal resources.

But these firms usually provide these services only for IT and data management teams. Such managed services could and should include expanded data profiling packages tailored for business users who are less technical but are business data savvy. Data profiling processes are almost always recurring, to support continuous efforts to gain more value from data and to ensure that data quality is reliable; as such, managed services have a 'natural' fit with the creation of tailored data management services for business users.

 

 

This post was brought to you by IBM for Midsize Business and opinions are my own. To read more on this topic visit IBM's Midsize Insider. Dedicated to providing businesses with expertise, solutions and tools that are specific to small and midsized companies, the Midsize Business program provides businesses with the materials and knowledge they need to become engines of a smarter planet.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective