Highly Competitive - software industry insights

Data Integration Services Evolution

 

 

With the increased usage of data analytics to help solve business problems and improve strategies, data integration has come more to the forefront as a critical component of overall technologies for data-savvy organizations. New approaches and technologies for data integration are coming into play to better meet what organizations need, especially to support increased business user access to data management itself.

Over the past several years, data integration services have continued to grow and evolve, thanks largely to the cloud. Tapping into SaaS approaches, the early entrants were data integration as a service and purpose-built or pre-built data integrations.

 

Data Integration -- as a Service

Data Integration-as-a-service (IaaS) became possible as more organizations made use of SaaS, cloud services and even B2B data exchanges. Business ecosystems began growing in size and complexity thanks to online connectivity, promoting ease-of-access for B2B relationships and interactions. As more data flowed in the cloud, the need and demand for integration picked up momentum. Originally IaaS functioned as an IT-oriented service, mostly in stand-alone instances. But IaaS instances were soon embedded in other cloud services and applications, as well as cloud infrastructure platforms.

The ability to quickly move data to where it is needed must evolve beyond cloud services that only provide basic APIs and web services that still require users to build connections. While leveraging APIs and web services for loosely-coupled integration processes, IaaS rapidly added more sophisticated capabilities such as virtualized data services, governance, REST-ful services, and agile interoperability at multiple levels.

Cloud-based IaaS has since progressed into Integration Platform as a Service (iPaaS). The data integration services evolution now involves comprehensive platforms that leverage cloud services for the creation and execution of integration processes or flows that connect to systems in the cloud and on premises -- within a single entity or across multiple organizations. Services include critical capabilities: data governance and management, metadata discovery, data quality, transformation rules, agile access to data sources – all delivered with automation and ease-of-deployment as important aspects of the overall platform.

 

Purpose-Built Data Integrations

As a more narrow approach to integration, purpose-built data integration is designed to meet a specific business need, with pre-defined capabilities bundled in the solution. Frequently purpose-built data integration comes as an appliance. Purpose-built data integration services in the cloud comprise an evolutionary step for making data integration services available to business users, to initiate integrations between specific sources or applications.

Cloud services for purpose-built integrations usually include data replication, migration, and synchronizations for customer data. Typically integrations involve specific SaaS applications like Salesforce.com and Netsuite as "pre-set" sources and targets.

These services work with focused use cases where business users do not have to make lots of decisions about how the integrations will be accomplished. Wizards are employed to walk business users through the process. However, at minimum, business users must have knowledge of the sources and targets, and basic understanding of the data integration task, to work confidently and accurately through the purpose-built approach.

 

Business Innovation from Data Integration Services

The evolution of data integration services must align dynamically to the business as it changes. Business users require rapid responsiveness to integration needs, through technology that is 'proactively' ready for many use cases that reflect expected and realistic business situations. Innovations for cloud-based data integration services contribute to optimal business performance and to the faster delivery of valuable data that frequently has a short shelf life of relevance.

Data integration services can tap into the rich value of the cloud and how it connects businesses through common services, with business processes providing contextual frameworks. Such data and process agility underpins must-haves for the competitive organization: always-on intelligence, new business opportunities, innovations in products and business models, robust collaborative processes for the ever-growing business ecosystem.

 

Image source: trendmicro.com

 

This post was brought to you by IBM for MSPs and opinions are my own. To read more on this topic, visit IBM's PivotPoint. Dedicated to providing valuable insight from industry thought leaders, PivotPoint offers expertise to help you develop, differentiate and scale your business.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting

 

Finding Meaning in Big Data Through Metadata

 

 

With the growing interest in big data analytics combined with the shortage of affordable experts and skilled practitioners for deploying and managing big data initiatives, many organizations are looking to managed services as a way to fill the need. More MSPs continue to develop sophisticated services for big data processing, along with all other aspects of data management and analytics.

Big data processing can function well as cloud-based, end-to-end managed services that integrate bi-directionally with on premises data centers and systems. End-to-end services can include streaming data flows, data management activities like data governance, and data visualization and analytics. Big data analytics on cloud infrastructures require top notch management of security and compliance, areas that MSPs are currently improving anyway to proactively move into new technology areas in demand for many companies.

Remember: big data doesn't just refer to high volume or fast-moving data that is important to large corporations. More companies of all kinds need to extract value from a wide variety of data sources – often these sources are difficult and messy for traditional data processing. These sources are also part of the "big data" analytics story and often yield valuable insight. "Messy" data sources include: multi-structured or unstructured data with highly variable formats and semantics like social media content, log files, and e-mail; and machine-generated data from medical devices, industry sensors, automated machinery and systems, and GPS.

 

Metadata Management Services

An important must-have capability for big data processing services is comprehensive metadata management. Metadata provides information about a data item, such as product, that uniquely describes that item. Product metadata can include product ID, product category, supplier ID, size or dimensions, and so on. A field like product ID is also a means for linking to other data sources, for integration purposes. Through metadata descriptors, we can talk about data items in common terms, outside the actual item, and take advantage of metadata to integrate and better understand disparate data sources.

Metadata management is one of the key areas of comprehensive information and data governance. Agile data governance tools are in demand and all sorts of data management vendors are chasing this business. As MSPs expand their presence in data management and analytics services, metadata management and data governance should be of great interest to them. Hopefully the availability of managed services in these areas will make comprehensive data management more affordable and available to organizations of all sizes.

 

What Metadata Does for Big Data Analytics

Metadata is not just about data integration and enterprise data warehouses; for other enterprise needs, metadata helps find data during data discovery, and points the way to interpret and use data correctly. Metadata can greatly streamline and enhance processes to collect, integrate, and analyze big data sources. Working with messy big data like multi-structured sources means that metadata is critical to understanding this data and to connect it to other data sources. Multi-structured data comes with many difficulties – information can be mined, but for it to have meaning and value, attributes like sentiment, purpose and context must be determined as well, and correlated with data sources for customers, products, and so on.

Traditional data sources, such as relational data, provide plenty of logical structure through more easily obtained metadata. However, big data often does not carry a lot of 'native' metadata, so metadata from external sources is essential to unlock meaning. Big data may have to go through certain analytics processes to construct the beginnings of metadata. Then this metadata is correlated with the metadata from other data sources to derive the most useful logical model. As big data processing evolves, new types of metadata may arise to meet the special circumstances of different kinds of big data.

 

Image source: Princeton Tutoring

 

This post was brought to you by IBM for MSPs and opinions are my own. To read more on this topic, visit IBM's PivotPoint. Dedicated to providing valuable insight from industry thought leaders, PivotPoint offers expertise to help you develop, differentiate and scale your business.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

 

Analytics Fortify Resilient Midmarket Companies

 

In the U.S. midmarket companies constitute a unique and highly fruitful business realm. Many midsized firms are purposefully not on a trajectory to become large enterprises. These firms are finding continuous success because they know how to operate well as midsized companies, and have advantages not available to much larger organizations. Lighter on their feet than larger enterprises, midsized companies can more quickly adapt to change and opportunities. And unlike small companies, the midmarket has stronger finances to fund growth and weather down times. Growth for the midmarket often comes from organic expansion – and understanding what it takes to engender it, including business model innovation and re-invention.

Midsized companies are much more in touch with customers and their own employees. They are truly interested in nurturing both of these relationships because they know that the right employees foster distinctive and positive interactions with customers. Midsized companies offer talented employees a better chance to stand out and drive innovative efforts, both for offerings and for how the business is run.

 

 

The most recent performance data from the National Center for the Middle Market provides more insight into the success of midmarket companies over time:

• Consistent with the past 12 months, a solid majority (63%) of middle market leaders continue to report improved overall company performance versus one year ago.
• Companies have sustained the rate of increase in year-over-year revenue growth as reported in the last quarter, with a mean total revenue growth of 6.6% for Q2 2014.
• While the proportion of middle market firms experiencing revenue growth consistently outpaces the proportion that added workers in the past year, middle market firms have maintained the significant increase in hiring reported in the first quarter of 2014. For Q2 2014, more than 40% of firms report increased employment compared to one year ago.
• Middle market leaders from all revenue segments continue to express higher levels of confidence in the global, U.S., and local economies. The percentage of respondents expressing some level of confidence in the local and regional economy is an impressive 77%.

A final statistic: 63% of midmarket companies plan to continue to invest in the growth of their companies. This last statistic is interesting since part of that investment continues to flow into the implementation of cutting edge technologies such as advanced analytics (including big data analytics) to help fuel growth, innovation and competitive advantage. There has been a steady increase in analytics solutions that are a good fit for the midmarket, as well as providing better access for less technical business users.

 

Analytics are not limited to the functions of operational performance, financial and risk management, and supply chain optimization. Midsized companies are especially interested in analytics in support of improved marketing and sales, with customer insight as a top priority. Midsized companies have competitive advantage over larger organizations when it comes to more quickly implementing the results of analytics processes, through decision-making processes and actions.

Analytics options that currently appeal to midsized organizations include reports and data visualizations that help all levels in the business make good use of data to recognize patterns of change, as well as new trends that impact target customers and markets, products and services, and revenue. Predictive analytics help midsized companies pull in new kinds of data to go beyond traditional BI to anticipate future and unexpected trends and changes. Big data analytics provide the means to use large datasets and non-relational data sources to improve and expand intelligence initiatives.

 

Midmarket companies seek many of the same outcomes that larger organizations want:

• Faster, more informed analysis for making better decisions, identifying compelling opportunities, and spotting market changes (good or bad)
• Continuous customer intelligence to improve customer experiences and relationships
• Timely insight for the development of products and services
• Business model innovation
• Operational improvement and reduction of risk

 

Image source: National Center for the Middle Market

                        (a collaboration between The Ohio State University Fisher College of Business and GE Capital)

 

 

This post was brought to you by IBM for Midsize Business and opinions are my own. To read more on this topic, visit  IBM's Midsize Insider. Dedicated to providing businesses with expertise, solutions and tools that are specific to small and midsized companies, the Midsize Business program provides businesses with the materials and knowledge they need to become engines of a smarter planet.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

 

Mobile Data and Security Challenges

     

Mobile platforms are flooding into enterprises, whether it's through BYOD "consumer" apps like Dropbox, or from internal development teams creating apps for on-the-go employees. Just like SaaS and Cloud solutions, we now have more data in silos. And the same goes for security issues, with the added kicker that an entire device full of data and other compromising information can be easily lost or stolen. Quite frankly, the very people who want mobile platforms available for work also present the biggest security risks to enterprise data assets.

The knee-jerk reaction of many enterprises has been to try to block or limit the use of mobile devices and applications. This is not an option – the floodgates are open. In fact mobility is critical in enabling a distributed work model for a global workforce. It's clear that not only do many employees want to work this way -- for many corporate roles, mobile empowerment is essential.

There is a deluge of mobile data being created by mobile apps, that is stored on devices or in the cloud. Many times this data must be integrated with on-premises systems, which introduces new proprietary APIs and challenges for integration processes. Employees, including upper management, also want all kinds of data from internal systems available on a mobile interface, including complex data visualizations. Data is definitely at risk as mobility proliferates:

• Loss or theft of smartphones, tablets, and laptops compromises corporate data and access to enterprise systems
• Many people mix professional and personal usage on a single mobile device, such as a single consolidated "Inbox" for all email accounts -- this opens the door to sending proprietary corporate information to and from non-corporate accounts, and even to the wrong recipients

• Mobility can lull people into forgetting whose data and information they are handling. Without physical corporate walls to remind them, they can inadvertently expose data and credentials

• Malware on mobile devices is on the rise with obvious implications for data security

 

Proactive Policies - Fluid Security

In anticipation of potential mobile security issues, enterprises need to lay out plans to manage the associated risk. First, upper management should develop a realistic strategy for mobility. Then data security and management strategies must be created that map into the corporate mobile strategy, and also align with corporate data governance practices. Finally the enterprise must put policies and practices into place, with straightforward education for employees about how mobility will work.

As with on-premises and cloud data sources, enterprises must proactively develop thorough data management and governance plans to secure the data sources themselves, as well as guidance for the bi-directional movement of data and integration processes.

A mobility security plan must be flexible to allow high volumes of interaction, but extensive enough to moderate risk and meet compliance standards. Enterprises can turn to advanced security tools to provide coverage for the internal and external sides of the mobile data equation. Mobile technologies are rapidly evolving along with employee usage – enterprises must keep pace by adopting ever-changing approaches to sophisticated protection and risk management.

 

 

 

 

 

Image source: IdroidWeb

 

 

This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet. I've been compensated to contribute to this program, but the opinions expressed in this post are my own and don't necessarily represent IBM's positions, strategies or opinions.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

 

 

The Journey to Insight and Action

..

There's no doubt that most companies are seeing massive increases in data assets, both internal and external. Beyond social content, enterprises encounter data explosions from business activities such as M&A, outsourcing initiatives, and highly diversified partner and supply chain programs. Healthcare, utility and telecom industries generate enormous stores of machine-generated data related to customer activities as well as to operational performance and 'product' pricing. GPS data continues to yield more and more customer and market insights for certain industries. And it's all quite overwhelming for many companies.

Midsized organizations are aware of the growing importance of extracting more value from data already in enterprise repositories, as well as continuously looking to external data sources. But to extract value from data, it's not about the data. The key focus should be on connecting data and analytics processes to what matters to the business, whether it's operational improvement, overall high performance, enhanced customer experiences, or innovative product development.

To help make the connection to desired outcomes, midsized organizations must understand the vital relationship between data and business processes, and must define the business cases and problems to be solved that have strategic and business-critical purposes. Value will be achieved from data when organizations identify processes and problems that impact revenue, business agility, competitiveness, and overall positive performance.

Strong data management initiatives are essential to ensure reliable, unified and timely data for key systems and processes. Such data must conform to how information is used by the business. Technologies play a crucial role in data management, but organizations must resist the temptation to think of data management as just a technical or IT initiative. Data management delivers best when it is directly tied to business opportunities, goals and objectives.

New processes must now come into play to understand how to:

• Validate data sources to ensure trustworthy and relevant information for analytics
• Verify and correlate analytics results with other intelligence sources to secure the right path for decision-making
• Use analytics results for deriving insights and reach decisions more quickly
• Take action on decisions in an agile and timely fashion while managing risk

 

The journey to continuously improved decisions and actions doesn't really start with data – it starts with what the organization wants to accomplish with its data assets. But the right data, business processes and analytics results can propel companies to better achieving goals and sustaining competitiveness. To make the journey to agility and high performance requires strong data management initiatives that combine analytics processes with cross-functional collaboration and practices that inject intelligence into corporate DNA for all purposes.

 

                      

 

 

 

 

 

 

 

 

 

  

 

 

 

 

 

 

Source: Accenture, 2012

 

 

Image source: Real Food Rules

 

This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet. I've been compensated to contribute to this program, but the opinions expressed in this post are my own and don't necessarily represent IBM's positions, strategies or opinions.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective

More Than Pretty Pictures – Visualizing Insight

 

...

            Source: customerthink.com

 

While more analytics tools, including data visualization applications, are becoming available at a wide range of cost-effective pricing and ease of usability, many midmarket enterprises continue to try to base forward-looking decisions on time-worn methods:

• Pulling analysis from backward-looking spreadsheets
• Waiting too long for tech teams to set up new BI processes
• Poring through the same industry and demographics stats that are available to their competitors
• Relying too much on 'gut feel' and 'that's how we've always done it'

Just like larger enterprises, midmarket companies need to be able to derive new views of data, especially data from disparate sources, to pull out deeper connections and relationships that have the potential to inform better insight and decisions. Data visualization solutions can help midmarket enterprises make sense of data in ways that communicate better understanding to more roles in the company. Visualizations can enable different business roles to more quickly understand and retain the intelligence revealed by the analytics results.

Graphs, charts and dashboards kicked off data visualization forms years ago and obviously are still in use. Now imaginative, almost art-like representations, such as social networking visualizations, are being added to analytics. But be careful not to confuse pretty for insightful or even accurate

Many pitfalls can exist along the way to usable and trustworthy visualizations. Providing context is essential for generating effective and accurate data visualizations. Context - and the dynamics of business - can make or break the value of a particular data visualization. Context, relevant data and awareness of potentially shifting business conditions all must be part of the process for asking the right questions that lead to usable results. Context also acts as the means to determine if the analytics results make sense.

Some complex datasets require complex visualizations that still may not clearly communicate analytical messages for business roles. There are times when companies will need to supplement data visualizations with other analytics, to present comprehensive intelligence for complex problems or questions in ways that lead to clear understanding.

The challenge for any analytics is to make sure that the results are accurate, timely and use the right information. Visual representations can be harmfully inaccurate by including the wrong data, excluding the right data, or by using faulty algorithms. Or the data could be the right stuff, but how the analysis is depicted may cloud the actual insights being presented. Knowing what works and what doesn't are essential to providing real business value.

New directions for data visualizations will continue to proliferate, along with new challenges and opportunities. Analytics results represented by data visualizations are only worthwhile if they add important intelligence, illuminate the meaning of data, and educate more roles in the company. Companies should never lose sight of why a data visualization was created in the first place. Real business value, trustworthy insight and continuous improvement should be the goals of most data analytics processes.

 

 

This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet. I've been compensated to contribute to this program, but the opinions expressed in this post are my own and don't necessarily represent IBM's positions, strategies or opinions.

 

About the author: Julie Hunt understands the overlap and convergence of many business processes and software solutions that once were thought of as "separate" – and how this impacts software Vendors and Buyers, as well as the strategies that enterprises implement for how technology supports the business and its customers. Julie shares her takes on the software industry via her blog Highly Competitiveand on Twitter: @juliebhunt For more information: Julie Hunt Consulting – Strategies for B2B Software Solutions: Working from the Customer Perspective