Why It Is So hard to Apply AI in Industry? Part II: Business Challenges
TL;DR: In the previous post, we addressed the technical difficulties of applying data science in the industry. In this second part of the series, we will discuss business challenges such as methodologies, stakeholder alignment, strategy, and cultural change.
We can highlight three main business challenges when applying data science in the industry:
1. Holistic understanding of the business problem being addressed
2. Strategic alignment between company areas and stakeholders
3. Cultural aspects
Let’s explore these challenges in more detail.
1. Holistic Understanding of the Business Problem Being Addressed
Although it may seem obvious that data science should be used to solve business problems, it is surprising how often, during an analytical model development project, it is necessary to revisit and remind stakeholders (and even data scientists) of the problem we are facing.
When we refer to a “problem,” we consider not only the impact on KPIs (Key Performance Indicators, such as metal recovery in a mining company’s flotation plant) but also the decision-making process.
While the main issues in the industry are generally well-known, and there is often a notion of the tangible impact they can have on a specific area or part of the production process and its respective KPIs, the biggest unknown is how the current decision-making process will be impacted by an algorithm.
Why is this delicate? Because decision-making will no longer be the same; it will be empowered and/or enhanced by an algorithm/model. This is critical and completely changes how we must approach a data project.
To facilitate understanding, let’s look at some common issues in the industry:
1. Unplanned downtime due to equipment failures, communication, or operational conditions
2. Inefficient process control, either due to the complexity of the production process or lack of know-how in adjusting control parameters
3. Operating the plant “blindly” for at least two hours, as it depends on the availability of laboratory analyses at intervals of at least two hours or more
Although these issues are known, as they strongly impact the production chain, the “new” decision-making process is often neglected. The picture below show this modern decision process, which is empowered by data and algorithms, which also amplifies our experience and judgment possibilities.

For example, if you are developing a predictive model that will predict 24 hours in advance that a piece of equipment (a centrifugal pump, for example) will stop, how do you imagine the decision-making process would be? Possible scenarios:
1. Mobilize the maintenance team to schedule an emergency stop (in the next 24 hours) on the equipment/production line and carry out an inspection or even a maintenance action
2. Leave a team on standby in case the equipment actually stops, so you and the team are ready to perform the repair
3. Send an email notifying all teams (maintenance, production, electrical, etc.) that pump X has a high probability of failure and that they need to “get moving”
In all the scenarios described above, decision-making involves more than one person, usually one or more teams. Notice the cost associated with decision-making (mobilizing people/teams to wait for a possible event). This is complex to resolve and plan, as well as act effectively, given the complexity of people, processes, and procedures involved.
However, it is unfortunately very common not to consider these aspects related to decision-making, as it involves cultural, political, and procedural change work.
The good news is that there are successful paths for effective decision-making when developing a predictive model for a production plant, as in the example cited above.
The most common case we see is empowering the operators of that production process and those responsible for operating/manoeuvring such equipment. They are instructed to perform some kind of preventive manoeuvre (even a field inspection) to verify if the problem is likely to occur or even to be prepared to stop the process and intervene. Of course, everything will depend on the level of confidence in the analytical solution developed and implemented.
Another important aspect, albeit more technical, is to consider working with a prediction window of about 10 minutes, or even 1 hour. This way, the cost of decision-making is “cheaper” (shorter mobilization time for people/teams) and there is less generation of false positives. Generally, in this type of analytical solution (anomaly detection models, for example), the larger the prediction window, the greater the number of false positives (as the model’s uncertainty increases).
In other words, how will humans in an industry make better decisions when empowered by algorithms? This point may seem obvious, but often the person who will receive the information to make a better decision is not considered during the project.
Not considering both “ends” of the problem—the business and how it is impacted, as well as decision-making empowered by an algorithm—is critical to the success of a data science project in the industry. Therefore, it is important to understand the problem holistically, not just one part of it. We will discuss the decision-making aspect further when addressing the third challenge—the cultural aspect.
Two interesting articles that I recommend reading, which discuss these aspects, are McKinsey’s articles:
• Breaking away: The secrets to scaling analytics
• Analytics translator: The new must-have role
2. Strategic Alignment Between Company Areas and Stakeholders
It is not uncommon to see automation, processes, engineering, and maintenance areas seeking their glory around a more data-oriented approach (machine learning, data science, etc.)…
I also worked at a steel company and know exactly what this arduous “adventure” is like. Difficulties in obtaining data, incorrect data, lack of time to dive into the analysis, lack of budget to have a partner to leverage you, etc.
Even with highly qualified professionals in the industry, engineers with master’s and doctorate degrees in AI (Artificial Intelligence), we see a great difficulty in all efforts of these people having a significant impact on the business. Why?
There is no strategic alignment within the company for AI and data initiatives to be considered key to transforming the business. Initiatives end up being a set of isolated efforts without the necessary alignment. See how simple questions below can identify if the company you work for has strategic alignment with data:
• Does the company have a CDO (Chief Data Officer) or some C-level representative for data on the company’s board?
• Does the company have a defined budget to invest in AI initiatives?
• Does the company already have a dedicated team with defined roles and career paths for data profiles?
If there is no indication of answers to the questions above in the company, chances are that it is not even considering placing its data as a fundamental part of transforming its business.
However, companies that understand the need for this strategic alignment (I’m not just talking about Big Techs like Google, Meta, TikTok, etc.) but industries with a CDO, budget, and more than five people focused on analytics (I know industries with more than 100 professionals dedicated to this).
3. Cultural Aspects
The third and final pillar for data to truly be transformative agents in a business, but no less important, is the cultural pillar.
Talking about culture is a somewhat complex and difficult subject, but the little I have studied and read about it shows that it is really difficult to work with and, as we know, often neglected. It is not even mentioned as an aspect to consider in a data project.
We talk about pipelines in Airflow, using an optimized hyperparameter search for an ensemble model, etc. But we almost never talk about how we will prepare the person who will receive this new information for their new way of making decisions. At what point in the project’s development will I put myself in the place of those who will receive all this novelty and actually understand if it is cool, usable, coherent, reliable, etc., so that I or the user can make a decision?
Therefore, start considering all these aspects at the beginning of a data project, mapping from the beginning how the user makes decisions today and how they will make them when the analytical solution is running.
I recommend the initial reading of: