Blog

TL;DR;Last week, aignosi proudly attended <a data-mce-href="https://exposibram2024.ibram.org.br" href="https://exposibram2024.ibram.org.br" title="EXPOSIBRAM 2024" target="_blank" rel="noopener">EXPOSIBRAM 2024</a>, one of the largest and most influential mining events in Latin America. Held at Expominas in Belo Horizonte, the event attracted over 60,000 visitors and 450 exhibitors, providing a unique platform for networking and exploring the latest trends in mining technology and artificial intelligence (AI) solutions.<h3>AIOps and AI in Mining: What’s Next?</h3>As we connected with industry leaders and innovators, one key theme stood out— AIOps and its role in transforming operational technology (OT) in the mining sector. At aignosi, we’re leveraging AIOps to integrate DataOps and MLOps, enabling AI scalability and operational efficiency. During our conversations at EXPOSIBRAM, many were eager to learn more about our progress and maturity in applying AI to real-world industrial challenges. Our platform, SIENTIA™, specifically designed for the industrial OT environment, was a central topic of interest. The discussions revolved around how our technology enables mining companies to implement low-latency AI models with robust data governance—solving one of the key challenges faced by the industry today.<h3>Strengthening Partnerships and Exploring New Opportunities</h3>EXPOSIBRAM was more than just an exhibition—it was a chance for us to engage in meaningful conversations about the future of mining, driven by AI and AIOps. In addition to discussing the latest technologies, we explored new opportunities for partnerships, collaboration, and further innovation.Many attendees were particularly interested in Aignosi’s current stage of maturity in the AI market, with a focus on how we help industries optimize operations through machine learning and AI-powered decision-making.<h3>The Future of AI and AIOps in Mining</h3>As we move forward, we are excited to transform the connections made at EXPOSIBRAM into long-term collaborations. Our commitment to pushing the boundaries of AI in mining and AIOps continues to grow, and we look forward to furthering our role as leaders in industrial AI solutions.Stay tuned as we continue sharing our insights and innovation journey, and for more on AI, AIOps, and digital transformation, check our upcoming posts! Want to know more about our journey?We share our startup journey here. Join us and let’s grow together.For more on AI and learning materials, please check our previous posts. Follow us on <a data-mce-href="https://www.linkedin.com/company/aignosi-brasil" href="https://www.linkedin.com/company/aignosi-brasil" title="Linkedin" target="_blank" rel="noopener">Linkedin</a> #Aignosi #Exposibram2024 #AIOps #AI #Mining #Innovation #Technology #OperationalTechnology #Industry40 #MachineLearning #DigitalTransformation

EXPOSIBRAM 2024: Driving Innovation in Mining with AIOps

Data fabric AI artificial intelligence powered analytics, data, data verse, big data, data science, business intelligence, data lake

We are excited to announce a significant achievement for aignosi: our paper on Operational Efficiency leveraged by AIOps approach for OT has been accepted for presentation at <a data-mce-href="https://www.abmbrasil.com.br/por/evento/abm-week-8-edicao" href="https://www.abmbrasil.com.br/por/evento/abm-week-8-edicao" title="ABM Week 2024" target="_blank" rel="noopener">ABM Week 2024</a>, one of the most important events in the industrial automation, mining, metalurgy and materials in Latin America.Taking place from September 3rd to 5th in São Paulo, ABM Week 2024 will bring together industry leaders and experts to discuss the latest technological innovations and trends. During the event, we will have the opportunity to share our insights on AIOps, an approach that integrates DataOps and MLOps to enable AI scalability in the industrial sector. Our focus will be on demonstrating how AIOps facilitates low-latency inferences while ensuring robust governance over data and models.This achievement reflects aignosi’s commitment to driving industrial AI and fostering innovation in the global industry. We are eager to participate in this highly significant event, which aligns with our mission to lead technological advancements in the sector.We look forward to heading to São Paulo and contributing to the future of industry!Conclusion:aignosi’s participation in ABM Week 2024 marks an important step in the company’s journey toward industrial innovation. We will continue to explore and present solutions that enhance efficiency and governance in the application of artificial intelligence on a global scale.

Presenting AIOps Insights at ABM Week 2024

Global Data Flow And Connectivity - East Asia (World Map Courtesy of NASA)

We have been selected for <a data-mce-href="https://programas.sebraestartups.com.br/in/aceleracaomadridlisboa2024" href="https://programas.sebraestartups.com.br/in/aceleracaomadridlisboa2024" title="SEBRAE’s International Acceleration Program" target="_blank" rel="noopener">SEBRAE’s International Acceleration Program</a>. Out of 150 startups, we made it to the top 15 finalists of the acceleration program. We will receive mentorship from SEBRAE-MG and <a data-mce-href="https://betaibrasil.com.br" href="https://betaibrasil.com.br" title="Beta-i" target="_blank" rel="noopener">Beta-i</a> experts. In November of 2024, the program includes visits to Madrid, Spain, and Lisbon, Portugal, featuring matchmaking sessions with investors, potential partners, and clients, all sponsored by SEBRAE-MG. The acceleration program concludes at the Web Summit event in Lisbon, one of the world’s largest innovation events, with over seventy thousand attendees. We will also participate as an exhibitor, as part of the program’s award. This acceleration program is crucial as it strengthens our commitment to internationalization, enhances our connections and partnerships, and expands our business opportunities. This is why aignosi was founded, and we are excited to continue growing on a global purpose which is to unlock industrial AI scalability.

Finalist in SEBRAE’s Acceleration Program

Business network concept. Human Resources. Group of businesspeople. Management strategy.

TL;DR: In the previous post, we addressed the technical difficulties of applying data science in the industry. In this second part of the series, we will discuss business challenges such as methodologies, stakeholder alignment, strategy, and cultural change.We can highlight three main business challenges when applying data science in the industry:&nbsp;1. Holistic understanding of the business problem being addressed&nbsp;2. Strategic alignment between company areas and stakeholders&nbsp;3. Cultural aspectsLet’s explore these challenges in more detail.<h3>1. Holistic Understanding of the Business Problem Being Addressed</h3> Although it may seem obvious that data science should be used to solve business problems, it is surprising how often, during an analytical model development project, it is necessary to revisit and remind stakeholders (and even data scientists) of the problem we are facing.When we refer to a “problem,” we consider not only the impact on KPIs (Key Performance Indicators, such as metal recovery in a mining company’s flotation plant) but also the decision-making process.While the main issues in the industry are generally well-known, and there is often a notion of the tangible impact they can have on a specific area or part of the production process and its respective KPIs, the biggest unknown is how the current decision-making process will be impacted by an algorithm.Why is this delicate? Because decision-making will no longer be the same; it will be empowered and/or enhanced by an algorithm/model. This is critical and completely changes how we must approach a data project.To facilitate understanding, let’s look at some common issues in the industry:1. Unplanned downtime due to equipment failures, communication, or operational conditions&nbsp;2. Inefficient process control, either due to the complexity of the production process or lack of know-how in adjusting control parameters&nbsp;3. Operating the plant “blindly” for at least two hours, as it depends on the availability of laboratory analyses at intervals of at least two hours or moreAlthough these issues are known, as they strongly impact the production chain, the “new” decision-making process is often neglected. The picture below show this modern decision process, which is empowered by data and algorithms, which also amplifies our experience and judgment possibilities.<figure id="figure-dfgix0fdqllzi9r3rk"><img id="dfgix0fdqllzi9r3rk" src="https://cdn.durable.co/blocks/5ffyKlsroh4urUPXRggKNSedpkuUnFyYMt34gXHGopEep2oP4tWuRcbaQKeYlMMN.png" alt="White cyborg robotic hand pointing his finger to human hand with stretched finger - ai artificial intelligence." title="White cyborg robotic hand pointing his finger to human hand with stretched finger - ai artificial intelligence." class="object-cover bg-gray-100 rounded-2xl md:rounded-3xl lg:rounded-4xl" style="aspect-ratio: 16/9; object-position: 50% 50%;" data-corners="large" data-aspect="16:9" data-caption="" data-positionx="50%" data-positiony="50%" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/5ffyKlsroh4urUPXRggKNSedpkuUnFyYMt34gXHGopEep2oP4tWuRcbaQKeYlMMN.png" data-mce-style="aspect-ratio: 16/9; object-position: 50% 50%;"></figure>For example, if you are developing a predictive model that will predict 24 hours in advance that a piece of equipment (a centrifugal pump, for example) will stop, how do you imagine the decision-making process would be? Possible scenarios:1. Mobilize the maintenance team to schedule an emergency stop (in the next 24 hours) on the equipment/production line and carry out an inspection or even a maintenance action&nbsp;2. Leave a team on standby in case the equipment actually stops, so you and the team are ready to perform the repair&nbsp;3. Send an email notifying all teams (maintenance, production, electrical, etc.) that pump X has a high probability of failure and that they need to “get moving”In all the scenarios described above, decision-making involves more than one person, usually one or more teams. Notice the cost associated with decision-making (mobilizing people/teams to wait for a possible event). This is complex to resolve and plan, as well as act effectively, given the complexity of people, processes, and procedures involved.However, it is unfortunately very common not to consider these aspects related to decision-making, as it involves cultural, political, and procedural change work.The good news is that there are successful paths for effective decision-making when developing a predictive model for a production plant, as in the example cited above.The most common case we see is empowering the operators of that production process and those responsible for operating/manoeuvring such equipment. They are instructed to perform some kind of preventive manoeuvre (even a field inspection) to verify if the problem is likely to occur or even to be prepared to stop the process and intervene. Of course, everything will depend on the level of confidence in the analytical solution developed and implemented.Another important aspect, albeit more technical, is to consider working with a prediction window of about 10 minutes, or even 1 hour. This way, the cost of decision-making is “cheaper” (shorter mobilization time for people/teams) and there is less generation of false positives. Generally, in this type of analytical solution (anomaly detection models, for example), the larger the prediction window, the greater the number of false positives (as the model’s uncertainty increases).In other words, how will humans in an industry make better decisions when empowered by algorithms? This point may seem obvious, but often the person who will receive the information to make a better decision is not considered during the project.Not considering both “ends” of the problem—the business and how it is impacted, as well as decision-making empowered by an algorithm—is critical to the success of a data science project in the industry. Therefore, it is important to understand the problem holistically, not just one part of it. We will discuss the decision-making aspect further when addressing the third challenge—the cultural aspect.Two interesting articles that I recommend reading, which discuss these aspects, are McKinsey’s articles:&nbsp;• <a data-mce-href="https://www.mckinsey.com/capabilities/quantumblack/our-insights/breaking-away-the-secrets-to-scaling-analytics" href="https://www.mckinsey.com/capabilities/quantumblack/our-insights/breaking-away-the-secrets-to-scaling-analytics" title="Breaking away: The secrets to scaling analytics" target="_blank" rel="noopener">Breaking away: The secrets to scaling analytics</a>&nbsp;• <a data-mce-href="https://www.mckinsey.com/capabilities/quantumblack/our-insights/analytics-translator" href="https://www.mckinsey.com/capabilities/quantumblack/our-insights/analytics-translator" title="Analytics translator: The new must-have role" target="_blank" rel="noopener">Analytics translator: The new must-have role</a><h3>2. Strategic Alignment Between Company Areas and Stakeholders</h3>It is not uncommon to see automation, processes, engineering, and maintenance areas seeking their glory around a more data-oriented approach (machine learning, data science, etc.)…I also worked at a steel company and know exactly what this arduous “adventure” is like. Difficulties in obtaining data, incorrect data, lack of time to dive into the analysis, lack of budget to have a partner to leverage you, etc.Even with highly qualified professionals in the industry, engineers with master’s and doctorate degrees in AI (Artificial Intelligence), we see a great difficulty in all efforts of these people having a significant impact on the business. Why?There is no strategic alignment within the company for AI and data initiatives to be considered key to transforming the business. Initiatives end up being a set of isolated efforts without the necessary alignment. See how simple questions below can identify if the company you work for has strategic alignment with data:&nbsp;• Does the company have a CDO (Chief Data Officer) or some C-level representative for data on the company’s board?&nbsp;• Does the company have a defined budget to invest in AI initiatives?&nbsp;• Does the company already have a dedicated team with defined roles and career paths for data profiles?If there is no indication of answers to the questions above in the company, chances are that it is not even considering placing its data as a fundamental part of transforming its business.However, companies that understand the need for this strategic alignment (I’m not just talking about Big Techs like Google, Meta, TikTok, etc.) but industries with a CDO, budget, and more than five people focused on analytics (I know industries with more than 100 professionals dedicated to this).<h3>3. Cultural Aspects</h3>The third and final pillar for data to truly be transformative agents in a business, but no less important, is the cultural pillar.Talking about culture is a somewhat complex and difficult subject, but the little I have studied and read about it shows that it is really difficult to work with and, as we know, often neglected. It is not even mentioned as an aspect to consider in a data project.We talk about pipelines in Airflow, using an optimized hyperparameter search for an ensemble model, etc. But we almost never talk about how we will prepare the person who will receive this new information for their new way of making decisions. At what point in the project’s development will I put myself in the place of those who will receive all this novelty and actually understand if it is cool, usable, coherent, reliable, etc., so that I or the user can make a decision?Therefore, start considering all these aspects at the beginning of a data project, mapping from the beginning how the user makes decisions today and how they will make them when the analytical solution is running.I recommend the initial reading of:<a data-mce-href="https://www.gartner.com/smarterwithgartner/learn-the-art-of-culture-hacking-for-culture-change" href="https://www.gartner.com/smarterwithgartner/learn-the-art-of-culture-hacking-for-culture-change" title="Gartner’s article The art of culture hacking " target="_blank" rel="noopener"> Gartner’s article The art of culture hacking</a> <a data-mce-href="https://targetteal.com/pt/cursos/culture-hacking/" href="https://targetteal.com/pt/cursos/culture-hacking/" title="the course on Culture Hacking by Target Teal." target="_blank" rel="noopener">Course on Culture Hacking by Target Teal.</a>

Why It Is So hard to Apply AI in Industry? Part II: Business Challenges

Robot And Human Hand Holding Jigsaw Puzzle

TL;DR: The technical challenges of data science in industry. The main reasons why applying data science in industry is challenging are:&nbsp;1. The journey data takes before it is available for analysis&nbsp;2. The design and conception of machine learning algorithms&nbsp;3. The scarcity and robustness of dataLet’s delve into these reasons.<h3>1. The Long Journey Data Takes Before Being Available for Analysis</h3>Anyone familiar with the reality of a typical modern industry knows that there is a long and arduous journey before data reaches the hands of an analyst, engineer, or even a data scientist. The diagram below illustrates a typical example of the journey data takes to get to an analyst.<figure id="figure-mj2tbk9tzslz66p0jl"><img id="mj2tbk9tzslz66p0jl" src="https://cdn.durable.co/blocks/16ygoVCafg8iEQ7rh7OGwdznmO7kokP1M2wJWC2VeDyqrCY4e41lomlfVFySL1nC.png" alt="How the data flows from sensor to a csv file an OT environment" title="How the data flows from sensor to a csv file an OT environment" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 1/1; object-position: 50% 40%;" data-corners="medium" data-aspect="1:1" data-caption="Typical data journey in a OT (Operational Technology) environment" data-positionx="50%" data-positiony="40%" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/16ygoVCafg8iEQ7rh7OGwdznmO7kokP1M2wJWC2VeDyqrCY4e41lomlfVFySL1nC.png" data-mce-style="aspect-ratio: 1/1; object-position: 50% 40%;"><figcaption class="body-normal text-gray-500 mt-2">Typical data journey in a OT (Operational Technology) environment</figcaption></figure>The data journey begins at the physical instrument, i.e., the sensor (temperature, pressure, etc.) that needs to be physically installed in the production process (the well-known, factory floor). The signal measured by the sensor is sent to a data and logic “hub” (in industry, known as the famous <a data-mce-href="https://en.wikipedia.org/wiki/Programmable_logic_controller" href="https://en.wikipedia.org/wiki/Programmable_logic_controller">PLC</a> or Programmable Logical Controller) via a 4 to 20 mA current signal. It’s important to note that this signal can suffer interference or even some “compression” when it needs to be converted from analog to digital. In this conversion process, <a data-mce-href="https://en.wikipedia.org/wiki/Quantization_(signal_processing)" href="https://en.wikipedia.org/wiki/Quantization_(signal_processing)" target="_blank" rel="noopener">quantization</a> of the signal occurs at the interface (<a data-mce-href="https://en.wikipedia.org/wiki/Programmable_logic_controller#Modules%5B3%5D" href="https://en.wikipedia.org/wiki/Programmable_logic_controller#Modules%5B3%5D">analog card</a>) that receives it and sends it to the “hub.” The figure below shows an example of the quantization phenomenon.<figure id="figure-40afne69klz66sqz6"><img id="40afne69klz66sqz6" src="https://cdn.durable.co/blocks/17Ig3CcC7kZdPzK4sqC1dTSGRuPAywb7VjmVyN0VjPeIsAGrjwBzy4oKEQWsYjr3.png" alt="How a quantized (or quantization) changes the original analog signal." title="How a quantized (or quantization) changes the original analog signal." class="object-cover bg-gray-100 rounded-2xl md:rounded-3xl lg:rounded-4xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="Quantized signal example" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/17Ig3CcC7kZdPzK4sqC1dTSGRuPAywb7VjmVyN0VjPeIsAGrjwBzy4oKEQWsYjr3.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"><figcaption class="body-normal text-gray-500 mt-2">Quantized signal example</figcaption></figure>Once the data is virtually available in the PLC, it is ready to be used by the PLC itself (which is basically an industrial “computer”) and also to be sent over the network to other interfaces/systems. That is, the data becomes available to another interface, the network card, which handles transmitting the data over TCP/IP using the famous and well-established industrial open communication protocol, <a data-mce-href="https://en.wikipedia.org/wiki/OPC_Foundation" href="https://en.wikipedia.org/wiki/OPC_Foundation">OPC</a>. This protocol is widely recognized and used, ensuring interoperability between different systems.From here, the data can pass through multiple interfaces, switches, and firewalls before actually reaching a server. Note that so far this data has not been stored in any database! We will talk about this shortly.Once the data reaches a server, it “enters” this machine through an application we call an OPC server. In other words, this application receives the data stream coming from the PLC and makes it available to be sent to more servers/applications via network communication using Microsoft’s well-known DCOM (Distributed Component Object Model), despite its information security issues. <a data-mce-href="https://en.wikipedia.org/wiki/Distributed_Component_Object_Model" href="https://en.wikipedia.org/wiki/Distributed_Component_Object_Model">DCOM</a> is still widely used for communication in Operational Technology (<a data-mce-href="https://en.wikipedia.org/wiki/Operational_technology" href="https://en.wikipedia.org/wiki/Operational_technology">OT</a>) environments.Nowadays, it is more common to find <a data-mce-href="https://en.wikipedia.org/wiki/OPC_Unified_Architecture" href="https://en.wikipedia.org/wiki/OPC_Unified_Architecture">OPC UA</a> (the standard protocol for Industry 4.0 applications) with security certificates, which mitigate cybersecurity risksAfter the data reaches a computer and is ready to be distributed, it is finally stored in a database. However, storing data in an industrial environment is not done simply using a relational database with SQL. Due to the need to manage the traffic, storage, and access to information in industrial environments, a specific system for this purpose was created, known as <a data-mce-href="https://en.wikipedia.org/wiki/Plant_Information_Management_System" href="https://en.wikipedia.org/wiki/Plant_Information_Management_System">PIMS</a> (Plant Information Management System).PIMS systems were developed to be a kind of “Hadoop” for industry, facilitating the storage of large volumes of data (mainly time series) in <a data-mce-href="https://en.wikipedia.org/wiki/On-premises_software" href="https://en.wikipedia.org/wiki/On-premises_software">on-premise</a> environments. Emerging in the 1980s, these systems revolutionized industry, and even today, there are industrial plants that do not have a PIMS system. It is important to note that, to enable the storage of large time series, PIMS systems also incorporated data compression algorithms.The most well-known compression algorithm is the <a data-mce-href="https://kb.elipse.com.br/algoritmos-de-compressao-de-dados-em-historiadores-de-processos-com-banco-de-dados-comerciais/" href="https://kb.elipse.com.br/algoritmos-de-compressao-de-dados-em-historiadores-de-processos-com-banco-de-dados-comerciais/">box/car backslope</a>, which allowed for the storage of a vast amount of data on 1 TB hard drives (considered significant storage space until about five years ago). This ability to store large volumes of data, combined with efficient information management, makes PIMS systems a fundamental solution for the industrial sector.However, as commonly known, solving one problem often creates another. Data compression also has its drawbacks, as well explained in <a data-mce-href="https://www.researchgate.net/publication/222687314_The_Impact_of_Compression_on_Data-Driven_Process_Analysis" href="https://www.researchgate.net/publication/222687314_The_Impact_of_Compression_on_Data-Driven_Process_Analysis">this article</a>.Finally, the data reaches its final destination, the PIMS. However, with current cloud technologies, data continues its long journey to a Data Warehouse or Data Mart.Regardless, it is from the PIMS that we perform data extraction of variables for exploratory analyses and predictive model development. It is in the PIMS that we query and export data to the well-known *.csv format.Note that, in industry, for data to be minimally available to the end user, it goes through a long journey where the probability of interference is significant. This is quite different from collecting and evaluating user click data on a web page to interpret engagement, for example, where the digitization and availability journey of the data for analysis is much shorter.Obviously, the goal here is not to compare journeys A and B but to highlight the importance of considering the long and complex path that data travels in an industrial environment before being available for analysis.Now, let’s address the next major challenge of data science in industry.<h3>2. Conception of Machine Learning Algorithms</h3>The second challenge of data science in industry is related to the conception of most machine learning algorithms.As mentioned by Andrew Ng <a data-mce-href="https://youtu.be/0jspaMLxBig" href="https://youtu.be/0jspaMLxBig">in this live session</a>, most ML (machine learning) algorithms were conceived and validated by academia and internet-based companies using reference data from digital world datasets and experiments. A notable example is the <a data-mce-href="http://yann.lecun.com/exdb/mnist/" href="http://yann.lecun.com/exdb/mnist/">MNIST DATABASE</a>, probably the largest and most reliable database of handwritten digits in the world, used in the architecture and validation of many deep learning algorithms.However, finding something similar for industry and manufacturing is practically impossible. Despite some small initiatives, nothing compares to the internet world. This is mainly because industries do not want to share their production process data due to intellectual property and brand protection issues. While curating and anonymizing this data would allow the scientific and professional community to improve research and developments in the industrial context, this practice is still in its infancy, as exemplified by the initiatives of <a data-mce-href="https://oceanprotocol.com/" href="https://oceanprotocol.com/">Ocean Protocol.</a>While I was working at <a data-mce-href="https://ihm.com.br" href="https://ihm.com.br">ihm stefanini</a> (back in 2018) and I managed to do something similar when one of our clients donated part of their data so we could make it available on Kaggle, anonymizing the data from a production process.<figure id="figure-70yp0h7z6lz66vtfk"><img id="70yp0h7z6lz66vtfk" src="https://cdn.durable.co/blocks/254KwFoa9hmQ36YmhNlXnwZQfa1AurkXqrPMavOGSdA8dcnB9KXG9MBcSUyNHL7U.png" alt="Kaggle industrial data set example." title="Kaggle industrial data set example." class="object-cover bg-gray-100 rounded-2xl md:rounded-3xl lg:rounded-4xl" style="aspect-ratio: 16/9; object-position: undefined undefined;" data-corners="default" data-aspect="16:9" data-caption="Industrial dataset on Kaggle example." data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/254KwFoa9hmQ36YmhNlXnwZQfa1AurkXqrPMavOGSdA8dcnB9KXG9MBcSUyNHL7U.png" data-mce-style="aspect-ratio: 16/9; object-position: undefined undefined;"><figcaption class="body-normal text-gray-500 mt-2">Industrial dataset on Kaggle example.</figcaption></figure><a data-mce-href="https://www.kaggle.com/search?q=industrial+in%3Adatasets" href="https://www.kaggle.com/search?q=industrial+in%3Adatasets">Link to industrial datasets on Kaggle</a>I receive messages almost daily from people around the world thanking us for the available data. It was then that I clearly realized the scarcity of this type of data for the global community.If there is not much data access, how do we design and validate algorithms? This becomes practically unfeasible or is done only by companies and private research where intellectual property is developed.In practice, we need to take these algorithms and try to adapt them to the industrial context, which often faces the third challenge I mention next.<h3>3. Scarcity and Robustness of Data</h3>Since the algorithms were not conceived considering the challenges of an industrial environment, as mentioned by Andrew Ng in the same live session cited earlier, we often observe a significant difference in the performance of the model obtained during development and the model in production. This occurs even in cases where the scientific process was rigorously followed and respected (correct data splitting, cross-validation, absence of data leakage, etc.). More details can be found in this important article <a data-mce-href="https://arxiv.org/abs/2108.02497" href="https://arxiv.org/abs/2108.02497">How to avoid machine learning pitfalls: a guide for academic researchers</a>.The main reasons for this are:&nbsp;1. Amount of data available (a classic problem in industry as well)&nbsp;2. Constant occurrence of drift (data drift and concept drift)&nbsp;3. Variations in production campaigns or changes in the operational point of the production process&nbsp;4. Sensor degradation and/or interference in the data journeyWe will not go into detail about each of these reasons in this post, but we can comment that, especially when creating anomaly detection models, it is common to have imbalanced classes, particularly regarding the positive class label of failure or anomalous behavior.Drift is also a problem that affects models in production. It is common to need to retrain models every 3-4 months.The most challenging aspect to deal with, due to the difficulty of control, is the third item, related to operational variations or production campaigns. Often, we cannot have this scenario in the training dataset or even never in the test dataset, meaning you end up with little visibility of the model’s actual generalization capability.It is also worth mentioning that, besides all the influences data undergoes, there is a need for instrument or sensor maintenance and recalibration. This can be expensive and is not always performed with the proper frequency. To make matters worse, it is difficult to determine how much error a damaged sensor is introducing into the real signal.<h3>In Summary</h3>The challenges mentioned in this post should always be remembered and considered when estimating the effort and risks of a data project in industry.In the next post of the series, we will address the business challenges that also impact the outcome of a data project in industries.

Why It Is So hard to Apply AI in Industry? Part I: The Technical Challenges

low-angle photography of man in the middle of buidligns

<h3>TL;DR</h3>This is our inaugural blog post! Just a brief summary about the moment of AI and how it can start impacting your business right now.<h2>Understanding the Role of AI in Business</h2>Artificial Intelligence (AI) is no longer a concept of the future. It is here, and it is transforming the way businesses operate, rioght now! From automating routine tasks to providing deep insights, AI has become a vital tool for companies worldwide.People are using AI to improve operational efficiency and make informed decisions. AI can analyze vast amounts of data quickly, helping businesses stay ahead of the competition. This technology is not just for tech giants; small and medium enterprises can also leverage AI to grow.<figure id="figure-wbofz4v72elyxiisja"><img id="wbofz4v72elyxiisja" src="https://images.unsplash.com/photo-1508243529287-e21914733111?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wyNjI5NjF8MHwxfHNlYXJjaHwyMnx8YnVzaW5lc3MlMjBBSXxlbnwwfHx8fDE3MjE2ODQ1Mzh8MA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080" alt="business AI" class="object-cover bg-gray-100 rounded-2xl md:rounded-3xl lg:rounded-4xl" style="aspect-ratio: 16/9;" data-mce-src="https://images.unsplash.com/photo-1508243529287-e21914733111?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wyNjI5NjF8MHwxfHNlYXJjaHwyMnx8YnVzaW5lc3MlMjBBSXxlbnwwfHx8fDE3MjE2ODQ1Mzh8MA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080" data-mce-style="aspect-ratio: 16/9;"></figure><h2>Optimizing Operations</h2>AI can streamline various business operations. For instance, in supply chain management, AI can predict demand and optimize inventory levels. This reduces waste and ensures that products are available when customers need them.In factories, it can predicte potential failures and anomalies which can lead to huge amounts of savings in terms of operational cost.In the logistics sector, AI can detect fraudulent activities and manage risks. This helps businesses protect their assets and build trust with their clients. <figure id="figure-25jve8j6hlyxiiy5a"><img id="25jve8j6hlyxiiy5a" src="https://images.unsplash.com/photo-1432888498266-38ffec3eaf0a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wyNjI5NjF8MHwxfHNlYXJjaHwxMXx8bWFya2V0aW5nJTIwYW5hbHlzaXN8ZW58MHx8fHwxNzIxNjg0NTQ1fDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080" alt="marketing analysis" class="object-cover bg-gray-100 rounded-2xl md:rounded-3xl lg:rounded-4xl" style="aspect-ratio: 16/9;" data-mce-src="https://images.unsplash.com/photo-1432888498266-38ffec3eaf0a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wyNjI5NjF8MHwxfHNlYXJjaHwxMXx8bWFya2V0aW5nJTIwYW5hbHlzaXN8ZW58MHx8fHwxNzIxNjg0NTQ1fDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080" data-mce-style="aspect-ratio: 16/9;"></figure><h2>Challenges and Opportunities tools. Data privacy and security are also critical concerns that must be addressed.</h2>However, the opportunities outweigh the challenges. By embracing AI, businesses can innovate and stay competitive. The key is to start small, experiment, and scale up as they gain confidence in the technology. A good way to speed up this journey is to use modern data platforms, like SIENTIA™, which reduce the time needed to put models into production. In other words, clients can now test AI much faster than before—what used to take an average of 9 months to put a model into production can now be done in minutes.<h2>Looking Ahead</h2>The future of AI in business looks promising. As technology advances, AI is becoming even more accessible and powerful. Businesses that adopt AI early will have a significant advantage over those that do not.AI is not just about technology; it's about people. It empowers individuals to focus on creative and strategic tasks, making work more fulfilling. By embracing AI, businesses can shape a better tomorrow for everyone. It offers numerous benefits, from enhancing customer experience to optimizing operations. While there are challenges, the opportunities are vast.By investing in AI, businesses can stay ahead of the curve and create a brighter future. The journey may be challenging, but the rewards are worth it. Let's embrace AI and shape tomorrow together.

Blog

EXPOSIBRAM 2024: Driving Innovation in Mining with AIOps

Presenting AIOps Insights at ABM Week 2024

Finalist in SEBRAE’s Acceleration Program

Why It Is So hard to Apply AI in Industry? Part II: Busin...

Why It Is So hard to Apply AI in Industry? Part I: The Te...

Shaping Tomorrow: The Future of AI in Business