Data refers to all the pieces of information that flows within the enterprise and its environment, such as information about employees, customers, and partners, which can be processed using computer systems.

This data may come from different sources:

• Files in diverse formats,
• Databases,
• Websites,
• Web APIs,
• IoT (Internet of Things) devices.

This data is important for businesses to help with their daily activities and their management. For example, analyzing data about customers, providers and partners is vital for the decision-making processes.

Nowadays, the volume and heterogeneity of corporate data makes it more difficult to use. We need specific tools capable of handling and analyzing this data, to identify trends and information understandable by employees and managers.

These new needs have given rise to a set of disciplines and techniques grouped under the umbrella called « Data Science ». A second group of skills has also emerged in recent years called « Data Engineering ».

Data engineers deal with specific aspects of the data application lifecycle such as data cleaning, data transport and the productization of the application, so that data scientists can focus on their main activities, such as writing code for data analysis or building machine learning models for the application.

As for fields that have seen their evolution towards maturity (one may think of the Web), the field of “Data” is currently in constant movement. These specific skills are now in great demand, there is feedback available, and the typical frictions that may exist daily in projects are also now known.

Today, a company does not simply decide to launch an application to provide some service. From the start, the team must think about different things and take them into account, such as:

• Which data sources to use (databases, APIs, and / or sensors)?
• Which storage mechanisms depending on the types of data involved?
• How fast does the data change?
• How would the result data?
• etc.

As we like to say, data is everywhere, and you could think of many ideas to make use of them. But, actually implementing an idea implies a long process of thinking about where you are at, what are your constraints, and which specialists you need to hire. You may have to start with a prototype to test things out, after which you will improve or rewrite the application based on the feedback received.