{"id":96,"date":"2023-07-14T14:50:00","date_gmt":"2023-07-14T14:50:00","guid":{"rendered":"https:\/\/baecke.io\/?p=96"},"modified":"2023-07-14T14:50:00","modified_gmt":"2023-07-14T14:50:00","slug":"ai-enterprise-governance-conversation-boards-unprepared","status":"publish","type":"post","link":"https:\/\/baecke.io\/?p=96","title":{"rendered":"AI in the Enterprise: The Governance Conversation Boards Are Not Prepared to Have"},"content":{"rendered":"<h2>The Governance Gap at the Top of the Organisation<\/h2>\n<p>Enterprise boards are approving AI strategies without a governance framework adequate to oversee the programmes they are approving. This is not a judgement about board competence. It is an observation about the mismatch between the speed at which AI deployment decisions are being made and the pace at which board-level governance capability for AI is being built.<\/p>\n<p>The pattern is recognisable from other technology governance moments. When cloud adoption accelerated in 2012 to 2015, boards approved cloud strategies without fully understanding the operational risk and regulatory implications. The governance gaps that resulted became visible through security incidents, compliance failures, and cost overruns over the following years. The organisations that had invested in board-level cloud literacy and appropriate oversight mechanisms navigated those incidents better than the ones that had treated cloud governance as an IT matter.<\/p>\n<p>AI is a more consequential version of the same dynamic. The decisions being approved involve AI systems that make or support consequential decisions about customers, employees, and operations. The liability for incorrect or harmful AI outputs is less clearly defined than the liability for most other technology failures. The regulatory environment is in active development, with the EU AI Act establishing obligations that enterprise boards need to understand but have not yet been adequately briefed on. And the pace of capability development means that the AI system the board approved six months ago may be significantly more capable, and therefore more impactful in its errors, than what was described at approval.<\/p>\n<h2>The Risk Dimensions Boards Need to Understand<\/h2>\n<p>The AI governance conversation that boards need to have covers four risk dimensions that are distinct from the risk dimensions of conventional enterprise software.<\/p>\n<p>Liability for AI-assisted decisions is the first. When a customer is denied a loan, rejected for a position, or provided incorrect information by a system that used AI to assist the decision, who is liable? The organisation that deployed the AI, certainly. But the liability exposure for AI-assisted decisions is not the same as for human-only decisions, and the existing liability frameworks that boards use to govern enterprise risk have not fully adapted to this. Boards need to understand, for each significant AI deployment, what the liability exposure is for incorrect outputs and whether the organisation&#8217;s governance and insurance frameworks adequately cover that exposure.<\/p>\n<p>Data privacy in AI systems is the second dimension. AI systems that process personal data have privacy implications that differ from conventional data processing in important ways. Training data may embed personal information in ways that are not visible in the trained model but can be extracted through adversarial queries. AI systems may infer sensitive personal attributes from non-sensitive input data. And the use of personal data for AI training purposes may not be covered by the consent the organisation obtained for the original data collection. Boards need assurance that AI systems processing personal data have been assessed against data protection requirements and that the governance framework adequately addresses the novel privacy risks AI introduces.<\/p>\n<p>Algorithmic bias in automated processes is the third dimension. AI systems trained on historical data may encode and amplify the biases present in that data, producing systematically different outcomes for different demographic groups. In contexts where AI-assisted decisions have regulatory implications, such as lending, hiring, or access to services, algorithmic bias is a regulatory risk as well as an ethical one. Boards need a mechanism for understanding whether the AI systems they have approved have been tested for bias and whether the testing methodology is adequate for the regulatory context.<\/p>\n<p>Security risk in AI-augmented workflows is the fourth dimension. AI systems that are integrated into operational workflows introduce new attack surfaces: prompt injection attacks that manipulate AI outputs to achieve attacker objectives, model poisoning that corrupts AI behaviour through manipulation of training data, and data leakage through AI systems that process sensitive information through third-party model endpoints. Boards need assurance that security risk assessment and control implementation for AI systems is as rigorous as for other enterprise software.<\/p>\n<h2>The Policy Decisions That Require Board-Level Ownership<\/h2>\n<p>Some AI policy decisions are operational and can be delegated to the management team. Others have implications significant enough that board ownership, or at minimum board awareness and explicit endorsement, is appropriate.<\/p>\n<p>The organisation&#8217;s approach to high-risk AI use cases, defined as AI systems that support or make consequential decisions affecting individuals, is a board-level policy decision. The EU AI Act defines high-risk AI systems by the domain and type of decision involved; boards should have a clear view of whether the organisation operates or plans to operate such systems and what the governance framework for them is.<\/p>\n<p>The organisation&#8217;s AI liability framework, including the insurance coverage for AI-related incidents and the contractual provisions in AI vendor agreements that allocate liability for AI failures, is a board-level financial risk decision. The CFO and general counsel need to be engaged in this analysis, and the board needs to approve the liability management approach.<\/p>\n<p>The organisation&#8217;s data governance framework for AI, including the policies on what data can be used for AI training and inference and how consent and data subject rights are managed in AI contexts, is a board-level data governance decision. This connects to existing GDPR obligations that the board has responsibility for, and it needs to be explicitly extended to cover AI-specific data processing.<\/p>\n<h2>The Monitoring and Accountability Structures That Make AI Governance Operational<\/h2>\n<p>Board-level AI governance requires monitoring and accountability structures that provide meaningful visibility without requiring deep technical expertise.<\/p>\n<p>The AI system register is the starting point: a maintained inventory of AI systems in deployment or development, with each system classified by risk tier, with the accountability structure, the governance controls in place, and the last review date for each. The register converts the abstract governance obligation into a manageable asset that the board can reference and that the management team is accountable for maintaining.<\/p>\n<p>The AI incident reporting mechanism is the second element. AI systems will produce incorrect, harmful, or unexpected outputs. The governance framework needs a defined process for identifying these incidents, assessing their significance, and escalating appropriate ones to board-level attention. The definition of &#8220;appropriate for board attention&#8221; is itself a governance decision: setting the threshold too high means the board does not have visibility into patterns of AI failure that should inform governance decisions; setting it too low means the board is receiving operational noise rather than strategic signals.<\/p>\n<p>The annual AI governance review, similar to the annual review that boards conduct for other significant risk areas, provides a structured mechanism for assessing whether the governance framework is adequate for the AI portfolio the organisation is operating, whether the risk tier assessments for individual systems remain current, and whether regulatory developments require governance framework updates.<\/p>\n<h2>The Question the Board Should Be Asking Right Now<\/h2>\n<p>If there is one question boards should add to their next technology agenda, it is this: for each AI system we have approved or are considering approving, who is accountable when it produces an incorrect or harmful output, and what is the process that would bring that failure to the board&#8217;s attention?<\/p>\n<p>If the technology leader cannot answer that question specifically for each significant AI deployment, the governance framework has a gap. The AI capability the organisation is deploying deserves governance that matches its consequence.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Enterprise boards are being asked to govern AI strategies and AI risk for technologies that most board members have limited experience evaluating. The governance conversation that needs to happen is not happening with the rigour the risk exposure demands.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-96","post","type-post","status-publish","format-standard","hentry","category-executive-briefings"],"_links":{"self":[{"href":"https:\/\/baecke.io\/index.php?rest_route=\/wp\/v2\/posts\/96","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/baecke.io\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/baecke.io\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/baecke.io\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/baecke.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=96"}],"version-history":[{"count":0,"href":"https:\/\/baecke.io\/index.php?rest_route=\/wp\/v2\/posts\/96\/revisions"}],"wp:attachment":[{"href":"https:\/\/baecke.io\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=96"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/baecke.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=96"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/baecke.io\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=96"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}