The AI Software Landscape
ModelOps - The Key Component in AI Deployment
Enterprise AI spans different departments, and along with new AI initiatives, it often incorporates existing AI or Big Data implementations from the last decade. Enterprise AI presents a unique opportunity for CIOs to consolidate existing data warehouses, data analytics, and business intelligence applications from various departments to find company-wide use cases that will heavily impact the bottom line. Each department and business unit should have its own applications and use cases for AI and given freedom and autonomy to use the most appropriate tools and techniques, which in the AI world are changing rapidly. But at a corporate level, there's a need for unified approaches to governance and operations that necessitates a centralized, disciplined approach to modelops.
Start by trying to think about production-sizing AI in the first phase of the project. The first phase is always “Understanding the business.” In this phase, they spend a lot of time pulling in essential people such as SMEs, IT, application developers, ml engineers, and data scientists. This team helps to come up with the use cases and the models. More importantly, they define key performance indicators of the models and plan out how the model will be moved into and maintained in production. Governance is the other critical discipline that requires special attention in AI projects. In regulated industries, good governance is essential. Enterprises that successfully operationalize their AI projects often have a better handle on data governance, model governance, and application governance than their peers. But, more importantly, models have life cycles. Unlike conventional software, models are not static. There’s the concept of data drift, mathematical drift, and business drift that necessitates continuous monitoring and updates to models. Keeping track of model behaviors, measuring them, and testing their production effectiveness are critical steps to successful Enterprise.
ModelOps Inspires Accountability
Enterprise AI projects touch many functional groups across the enterprise, including the data scientists who create models, the data teams that manage the data, the development and operations teams, the governance organization, and the business unit that sponsors the model's development to meet a business KPI. These different groups tend to operate in silos, which creates friction and slows the process of moving models from the lab into production.The role of ModelOps is to provide an efficient and transparent framework for operationalizing AI across the enterprise. It offers the opportunity to manage business accountability, share it across groups, and bring stakeholders such as compliance, finance, operations, and other business unit functions closer together to impact business outcomes.Bailey says, “One thing the panel was very clear about is that ModelOps is more of a business accountability capability than a technical capability. It’s certainly is very technical, but it has to allow for model life cycles, integrate into existing compliance programs, and be a part of the risk management program. For example, there have to be audits, approvals, governance functions for models to be in production in the banking industry. So, it’s clearly a business accountability function that reaches across the enterprise..”
Change the Game
For large enterprises that operate in ever-changing business environments, implementing enterprise AI means learning to change with the business environment. The critical competitive advantage in any market is speed. If a firm can quickly change their model adjusting to the business environment, there can potentially be exponential business advantage just being the first in the marketplace to adapt.Bailey says, “Many of these models have to be refreshed quite regularly, sometimes on a daily basis in terms of new data that the markets have seen. Being responsive to regime changes such as when the conditions in the market change, and starting the process of replacing the models according to that change is very useful. It is a case for automation when you need the speed, and there can be potential legal implications. You want it to be automated.”
New Roles Emerging in Enterprise AI
Along with data scientists and machine learning engineers, two roles emerge as central to bringing enterprise AI into production: The Enterprise AI Architect and the Model Operator.Model Operators are people who monitor hundreds or thousands of models in production. They have SLAs, and they have benchmarks. They take care of problems and escalate according to procedures. Their job is to make sure that models run smoothly in production.On the other hand, the Enterprise AI Architect is becoming the central liaison for different stakeholders to receive the big picture perspective on Enterprise AI projects. They are the people who are designing the life cycle for every model. They understand the business impact, stakeholder needs. They can translate these impacts and needs into sound model risk management plans in production. They are the go-to for data scientists, machine learning engineers, dev-ops, and data ops groups to define processes and systems that enable them to collaborate easily and effectively.Bailey says, “The role of the Enterprise AI Architect is really new. The EAIA is responsible for orchestrating the team around critical assets including models, their production life cycles, their KPIs and metrics, approvals, regulatory reports, retraining and refresh requirements, and operational requirements for, say, deploying a model into a particular cloud infrastructure. The Enterprise AI Architect articulates life cycle, either as a series of documents or as a set of functional automations using a tool like our ModelOp Center.“Given its central importance in operationalizing AI initiatives, this Enterprise AI Architect is becoming a highly visible role as enterprises strive to be AI-driven.
The CIO Becomes the Chief Innovation Officer
Due to the business impact of enterprise AI initiatives, the CIO is increasingly able to impact the business bottom line, not by reducing costs, but by developing a competitive advantage.In many organizations, the information technology group has played defense, focusing on cutting costs while the business units take the lead in driving innovation. With enterprise AI initiatives replacing legacy systems and consolidating data for governance, innovation becomes the information technology mindset. A new generation of CIOs are emerging to take on the challenge. They are people who are more creatively minded and can navigate disruptive changes to drive business advantages.Bailey says, “Information is increasingly the core proprietary asset for any company. When the CIO’s organization can automate operations and governance and focus more on the top line, then it’s a huge opportunity.”
ModelOps Provides the Foundation
The ecosystem of algorithms and offerings from various professional service firms has dramatically expanded. Our AI intelligence will become more and more adaptive. Therefore, ModelOps is just the beginning of the journey. At various stages on the AI Journey there will be challenges and complexities that enterprises will need to overcome.We are building complexity, with systems that can produce new decisions and provide insights by themselves, there will be a new set of metrics that will measure these systems' effectiveness and govern these models. It merely takes mindset changes across businesses, tools, and processes to enable the operating environment.