These guidelines was written for The Dispute Resolution Board Foundation and first appeared here.

Introduction

The use of Artificial Intelligence (AI) is nothing new. Back in 1950, Alan Turing posed the question: “‘Can machines think?” (1)

Fast forward to 2024 and AI is everywhere. It is not the future; it is the here and now.

The use of AI, now widely seen in Generative AI (Gen AI) tools and chatbots (2), is increasingly common across the construction and engineering, as well as the legal industry. The uses and applications of AI are multiplying daily, and many would argue its use in certain tasks already exceeds average human capabilities. This evolution and growth is constant and will likely continue into the near future.

AI is an advanced and complex tool, and in its current form, its use requires advanced knowledge in the subject matter to which the tool is applied, as well as prior knowledge and training on the tool itself, for effective and safe deployment. Whilst there is little doubt that AI tools, including Gen AI tools, have the potential to increase the efficiency and productivity of users, their use comes with many risks and challenges. The DRBF recognizes that DB practitioners and those who use DBs need to understand not only the potential benefits and opportunities that come with the use of AI, but also how to mitigate the bias, risks and limitations that may arise.

It is important to understand that the use and application of AI will vary across jurisdictions, organizations, and even amongst DB members themselves. The purpose of this Practice Guide is to help DB members, where AI is used, provide a framework for its responsible deployment.

The key principles that DBs need to consider before using AI are:

• Accountability.
• Confidentiality & Security.
• Ethics; and
• Transparency

The Purpose of this Guide

The purpose of this guide is to explain how AI, and in particular Gen AI, it is likely to be used in the DB Process and how it should be managed. Nothing in this guide is meant to replace the current best practices and procedures. Rather, the goal is to show DB practitioners how the well-established guidelines and best practices fit and should apply to the use of AI.

The use of AI will likely occur on all sides of the DB. The parties working with the DB, for example the project owner and the main contractor, are increasingly likely to use AI to track progress, update and model schedules, review recovery/acceleration plans, review design changes/value engineering options, allocation resource and monitor costs. Some may even carefully monitor correspondence for possible claims and watch the sentiment of the parties when they communicate with each other. Moreover, AI has the ability to identify clashes, or design errors at an early stage.

Construction projects are characterized by complex and voluminous documents. AI tools have the ability to review and summarize these documents, thereby increasing their usability, minimizing losses, and improving efficiency. These tools not only have the potential to predict the next steps or problems in real time, aiming to avoid problems, but also to capture associated costs and impacts. Many of the claims, requests, reports, and analyses routinely submitted to a DB, if not now then in the very near future, will be, at least in part, created with the help of AI tools.

While AI can help identify and create submissions, it can also, by its nature and intended ability to identify and capture, impose additional, previously uncontemplated burdens on a DB. For
example, it is possible to envisage a future where, on some projects, the regular site visits and an occasional consultation will be potentially replaced by the need for what may feel like full time engagement, as the project data, including real-time live reporting, is updated on a daily basis.

With respect to visits themselves, drones, sensors, and smart cameras, static and mobile, some with 3D and AI capabilities, provide many more opportunities for the DB to “visit” the project site virtually and observe progress as well as evaluate conditions. Where this is available, DBs must always analyze carefully whether the time and cost efficiencies of a virtual visit outweighs the considerable benefits to the dispute avoidance process of in-person site visits, communication, and engagement.

The question which arises, and is the subject of this guide, is whether a DB, who has received complex and voluminous claims, can itself rely on AI technologies to read, summarize, digest, and evaluate the claims it receives? For example, can the DB utilize AI to review correspondence and submissions to create meeting agendas? Can the DB use AI tools to review delay and quantum claims? Can the DB request original, raw, non-expert adjusted/prepared for claim purposes, project schedules and data to perform its own analysis? The answer to all these questions is “it depends.”

Ultimately, it is up to a DB to determine its level of comfort with utilizing AI tools. Our hope is that the following sections will help a DB practitioner, regardless of the role, to be prepared,
vigilant and most importantly to apply new technologies properly to solve age old construction problems.

Definitions

There is no agreed definition of AI from country to country or even law to law within the same county as specific legislation is introduced to address narrow problems. However, Article 3(1)
of the new EU Artificial Intelligence Act 2024 provides a good starting point by defining AI as:

“a machine-based system designed to operate with varying levels of autonomy that may exhibit adaptiveness after deployment that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments.”

This general lack of any agreed definition, together with the possible differing levels of understanding (if any) of AI means that the first thing a DB must do, when introducing discussion about AI is to ensure that everyone understands what is being discussed and uses the same agreed terminology moving forward.

It is helpful to look at accepted characteristics of AI, which help distinguish it from the other forms of advanced data applications that already exist:

• Adaptivity;
• Ability to Create New Content (for example, Gen AI); and
• Degree of Autonomy

The DRBF has included with this Practice Guide, a Glossary detailing some of the more common terms and definitions.

Permitted Use: Contract Documents and Jurisdiction

At the outset, if AI tools are being considered for use during the DB process, parties and the DB should determine whether the use of AI is permissible under the contract (including the DB Agreement and any associated DB Specification or DB Rules/Operating Procedures), and relevant legislation. This may include, for example, GDPR regulations which require the secure handling of sensitive information and the right not to be subjected to a solely automated decision producing legal or similarly significant effects. (3) The DB, once constituted, needs to evaluate whether it can use and rely on AI tools, to what extent, and under what conditions, and to discuss same with the parties. There may be some applications of AI where the DB may consider that it does not need to disclose its use, for example, if it is using ChatGPT or another platform to “google.” In the same way it would do with the internet, the DB must check any information it relies on. In other applications, where AI is influencing or informing a DB’s decision, disclosure of its use is more likely. Either way, apart from a general appreciation of
the use of AI, the parties and the DB must also consider whether the specific AI tool being considered is permitted.

It may be necessary for the relevant documents to be amended to allow the use of AI and to establish the conditions in which that use is permitted. The biggest traditional concern with any new technology is unintended consequences, which in the instance of a DB requires full disclosure and communication with all parties. A lot of the information submitted to a DB will be confidential, proprietary, and possible key to a party’s competitive advantage. The quickest way to lose a party’s trust in the DB process, or to cause a challenge to a DB’s findings, is to mishandle sensitive information.

In modifying contract documents, the parties and DB must take care to ensure that all legal requirements and confidentiality are observed.

Whilst one of the benefits of some AI-powered tools is document review and the ability to analyze large volumes of documents quickly, this is no substitute for the DB’s own review of project documents between meetings between regular site meetings.

At the end of the day, the DB is responsible for its processes and its decisions, regardless of whether it uses AI or not. If the DB utilizes AI, and relies on its output, the DB is responsible
for that content.

Confidentiality and Security

Canon 2 of the DRBF Code of Ethical Conduct states that:

“Board members must ensure that information acquired during the term of the Dispute Board remains confidential and must not be disclosed, unless such information is already in the public domain.”

A DB must at all times protect the confidentiality of the project, parties, and all site data.

Many of the AI tools that are available are not private and confidential (4) which means that any information or data that is put into them will immediately become publicly available and may in fact be utilized to train the AI tool and may modify its future responses. This includes not only the documents one may upload to it, but also any questions asked, or instructions given. The AI tool may use that information to reply to queries from others.

It bears repeating that any data which a DB loads into an open AI platform, for example a party’s claim including cost data, will be ingested into that platform and potentially available to the general public, including the party’s competitor, if the platform is not private and secure.

Unless the DB is certain that information is publicly available, and unless it is sure that the data will remain confidential to it and its own server, a DB should not upload the information into a public or open AI tool or chatbot. The DB should be careful too, using any “translation” service, to anonymize any request.

The DB should carefully consider the vendor hosting the AI platform and their revenue model. Paid subscriptions are generally more secure than simply using a public AI platform, though not always. The DB should ensure that it understands the limits of its subscription and any additional settings required to ensure security, and if different subscriptions offer different levels of data protection.

Confidential information, at the absolute minimum, includes the name of the project and parties, addresses, financial details, the names, and addresses of individuals, or potentially, the project documents such as schedules, drawings, cost reports, correspondence, and meeting minutes. It is important to be mindful of and comply with what the contract documents or applicable legislation considers confidential.

Sometimes smartphones, apps, and laptops, even for the same AI tool, require that one provides a separate set of permissions for each device. The DB should pause, check, and understand what permissions it is enabling. If the DB does not do this, it may inadvertently give the AI tool access to information on one of its devices. This applies to all software and digital tools, not simply AI tools.

It is better to use and access AI tools on work devices, not personal ones. Also, the DB should use its work email address/account not a personal one. Often tighter security protocols and software are deployed on work devices and accounts over personal ones.

Understanding AI prior to use

From the perspective of a DB, the use of AI should only be considered if it is confident that it understands how the tool works. What does the DB actually know about the algorithms and
data sources used by the AI tool in question?

The DB should ask itself:

• What are the benefits to the project of using AI?
• Are there any risks in using the particular AI tool?
• Is the use of AI a fair and reasonable substitute for human expertise?
• Can the use of AI, objectively, support or augment human expertise?
• Is the DB able to explain to the parties the benefits of using AI?
• Do the parties agree to the DB’s use of AI?
• For what specific tasks does the DB want to use AI?
• Will there be limitations on the use of AI?

AI is only as good as the data inputted into the system in question. There are many examples of AI, particularly GenAI, getting things wrong, making things up and/or hallucinating. Hence, it is important to carefully consider both what sources of data the AI tool is drawing from, what data has it been trained on, as well as its outputs.

The DB should always remember the value of experience and education. It should keep up to date with developments and best practice. The use of AI is no substitute for intelligence, judgment, particularly ethical and moral judgement, deep contextual understanding, and common-sense reasoning.

The DB must stop, think, and not get carried away by all the new and exciting features of the latest update.

The DB should interrogate any product updates. Are there any changes to the confidentiality and security of the system? Is its data as secure as it was under the previous update? Does it
need to update its permissions?

The DB must observe that AI is under rapid development and users should consider putting in place procedures and guardrails to monitor and evaluate the implication of those developments.

Accountability: Transparency, Decisions, Recommendations and/or Reports

AI tools have the potential to analyze and surface insights from large volumes of data, generate text and make predictions in response to patterns found in data. It is important to stress that the technology (and society) is not yet at the point where an entire Decision can or should be entirely produced by the machine alone: but watch this space.

Canon 4 of the DRBF Code of Ethical Conduct notes that:

“DB members should apply their own expertise and experience to the resolution of any dispute.”

A DB member is always responsible for their Decisions, Recommendations and/or Reports.

The DRBF does not consider that a DB should contemplate delegating any part of the decisionmaking process to any form of GenAI chatbot, or similar tool. Moreover, prior to utilizing GenAI or similar technology the DB must confirm that such use and its extent, is clearly and unambiguously disclosed to all parties, is accepted by all parties, and does not violate the contract documents or the governing law. (5)

If the DB intends to use AI tools to assist in the preparation of the content of its Decision, then it must be transparent and open about that use before actually starting. Explain what tools it intends to use and the role they will play. Emphasize that the use of AI is a tool to aid, not replace the DB’s own judgment. Most importantly, regardless of how the DB uses AI, it must make sure to check and verify that the results are accurate, whether it is a summary of a large document or an analysis of a schedule.

If the DB is using AI for research purposes, it should remember:

• To ask the right question.
• To think carefully about the question or prompt it is asking. The usefulness of the answer depends on the AI tool having context and detail from which it will draft the answer.
• The DB cannot currently rely on the accuracy of information provided by AI tools. It may be misleading or incomplete.
• There are many examples of AI hallucinating or making things up, sometimes simply in an attempt to be helpful and answer the question posed. This includes inventing articles, quotations, or legal cases.
• To think about copyright and any possible copyright infringement.

The DB should therefore take careful steps to review and verify any information obtained through the use of AI before integrating it into its Decision, Recommendation or Report.

One of the potential strengths of AI comes in document review and heavy data analysis. AI tools can analyze large volumes of documents quickly. This could potentially save time (and cost) when it comes to the preparation of Decisions, Recommendations and Reports. Again, the DB should be transparent and clearly explain to the parties if and how it intends to utilize AI and why – confirm that its proposed use of AI is acceptable to all parties.

The DB should keep records that explain its decision-making process, including the extent to which AI was used.

Bias Mitigation & Ethics

An AI tool is only as good as the quality of its input data, the nature of algorithms and the data it has been trained upon, and the ongoing way in which it learns and assimilates new data. This means that the information generated by the AI tool may reflect errors or biases in that training data.

The DB should therefore always take into account ethical considerations to ensure fairness and prevent discrimination. This includes:

• Considering, where appropriate, implementing measures to identify and mitigate biases in AI tools.
• Always reviewing AI-generated content carefully and critically.
• Ensuring that any use of AI aligns with ethical principles and professional standards governing the use of Dispute Boards which may be published from time to time.

Regardless of the DB’s use of AI (in whole, in part or not at all), the DB must always act in accordance with the principles of fairness, impartiality, and due process throughout the Dispute Board process.

Site Visits and Project Documents

The DB should raise the use of AI with the parties as soon as possible.

It is important to understand the extent to which, if at all, AI or other automated processes will be used to document progress on the project and what other, if any, common project tools are in place.

Certainly, the DB should discuss with the parties the availability of and access to real time video and sensor technologies to allow the DB greater access to and understanding of project conditions. Autonomous camera rigs, drones, statis cameras and virtual reality are just some of the tools that could be available to the DB for its use and consideration in making decisions.

One of the many benefits of site meetings are that they provide a means to encourage communication (sometimes simply ensuring the parties are talking to each other). Communication is a prime component of the dispute avoidance toolbox.

AI will not assist in promoting communication. In fact, the use (or over-reliance) of AI, might rightly be viewed as a factor which stands in the way of this key project management tool.

Dispute Board Hearings

If a DB intends to use AI as part of the Decision or Recommendation-making process, then in the interests of transparency and fairness, the DB should communicate these intentions at an early stage of the Referral process, if this has not already been made clear at earlier site meetings.

Especially when there are complex delay and quantum issues that need to be considered, parties often deploy the use of consultants or experts. These experts will increasingly make reference to automated processes and the use of AI tools.

Exactly the same principles should apply to these experts, and the use of AI during referrals by the parties, as they do to DBs themselves. Hearings and the referral process must be carried out in a fair and open way. Whilst AI can be a supportive tool, which aids the parties in putting a case together, and may help explain more clearly what has happened, transparency is of the paramount importance.

The DB must ensure that it understands the way in which a party, or their expert, has made use of AI tools. If the DB does not understand, it must ask and not become blinded by the science.

The ICC commented in a Guide entitled: Construction Industry Arbitrations: Recommended Tools & Techniques for Effective Management of Arbitrations:

“Whilst it is for the parties to determine the method, they will use to present their cases, when analysis and quantification of delay and disruption is concerned, as it often is, arbitrators in construction cases should appreciate that the analyses, graphs and charts are only as accurate as the facts on which they are based.”

Most construction contracts stress the importance of using contemporaneous records to make good claims. AI should not be used to, or allowed to, reduce the importance of understanding what actually happened.

One of the current strengths of AI is document review, and the ability to analyze large volumes of documents quickly. There may be advantages for all parties, including the DB, to adopt its use, particularly when dealing with quantum. This can be done, provided it is done openly and with the agreement of the parties.

Translation

AI often works quietly in the background. If the DB is using a computer system for the translation of documents or even consecutive or simultaneous translation during a hearing, it must ensure that the DB and the parties agree about the system to be used. Any agreed system should be tested prior to a decision to use them. In addition, note that open platforms such as Google Translate are not secure or confidential.

Conclusion

The use of AI is here to stay. It is rapidly evolving and if used with care, it can assist a DB in their role. However, AI must be viewed only as an “assistant” to the DB: it is an enabling and supporting tool, not a substitute for human judgment and experience. If proposing to use AI, DBs must first understand its risks and limitations and be completely open and transparent in advance about how it will be used.

Acknowledgments

The DRBF is grateful for the contribution of lead developers Jeremy Glover, Oskar Ljungqvist, Denis Serkin, Dr Stacy Sinclair, and contributors Graham Easton, Samuele Maria Semi and Ann Russo.

______________________________________________________________________________________________________

Glossary 

AI chatbots
A chatbot is a domain-specific conversational interface that uses an app, messaging platform, social network, or chat solution for its conversations. Chatbots vary in sophistication, from simple, decision-tree-based marketing stunts to implementations built on feature-rich platforms. They are always narrow in scope. A chatbot can be text- or voice-based, or a combination of both. (6) Many virtual assistants, such as ChatGPT and Gemini are AI-driven. They use techniques like natural language processing (NLP) and natural language understanding (NLU) to analyze and respond to user inputs. (7)

Artificial Intelligence
Artificial intelligence (AI) applies advanced analysis and logic-based techniques, including machine learning, to interpret events, support and automate decisions, and take actions. (8)

Black Box Syndrome
Black box models are created directly from data by an algorithm, meaning that humans, even those who design them, cannot understand how variables are being combined to make predictions. Even if one has a list of the input variables, black box predictive models can be such complicated functions of the variables that no human can understand how the variables are jointly related to each other to reach a final prediction. (9)

Generative AI
Generative AI refers to AI techniques that learn a representation of artifacts from data, and use it to generate brand-new, unique artifacts that resemble but do not repeat the original data. These artifacts can serve benign or nefarious purposes. Generative AI can produce totally novel content (including text, images, video, audio, structures), computer code, synthetic data, workflows, and models of physical objects. (10)

Hallucination
Hallucinations are completely fabricated outputs from large language models. Even though they represent completely made-up facts, the LLM output presents them with confidence and authority. (11)

Large Language Models
These are deep learning algorithms that can predict and generate text and other content based on knowledge trained from broad sets of data. These models can perform tasks such as recognizing, summarizing, or transforming information, which creates many opportunities for application. LLMs can be used to generate more direct, humanlike answers to search engine results, perform swift data analysis, create meeting transcripts, summarize calls, generate images and videos or better understand life science elements like proteins. (12)

Training Data
Training data is an extremely large dataset that is used to teach a machine learning model. Training data is used to teach prediction models that use machine learning algorithms how to
extract features that are relevant to specific business goals. (13)

______________________________________________________________________________________________________

(1) Computing Machinery and Intelligence, published in 1950. Co-Pilot, the AI Chatbot associated with Microsoft Bing suggested that a reference to Alan Turing’s “visionary” ideas would be an ideal opening to this Guide. For the avoidance of doubt, that was the first and last time any form of Generative Artificial Intelligence has been asked to assist in the drafting of this guide.

(2) For example, ChatGPT, Google’s Gemini, Anthropic’s Claude as well as Microsoft’s Co-Pilot.

(3) Article 22, EU General Data Protection Regulation (GDPR) 2016/679

(4) It is important to note that even if you are paying for an AI subscription, this does not mean that it is a private and confidential platform. The terms and conditions of the platform must be considered carefully.

(5) For example, Article 22 of the GDPR

(6) ‘Definition of Chatbot – Gartner Information Technology Glossary’ (Gartner) accessed 23 April 2024.

(7) Tim Keary, ‘What Is a Chatbot? – Definition from Techopedia’ (Techopedia.com) accessed 23 April 2024.

(8) Artificial Intelligence (Ai)’ (Gartner) accessed 23 April 2024.

(9) Cynthia Rudin and Joanna Radin, ‘Why Are We Using Black Box Models in AI When We Don’t Need To? A Lesson from an Explainable AI Competition’ (2019) 1 Harvard Data Science Review accessed 23 April 2024.

(10) ‘Definition of Generative AI – Gartner Information Technology Glossary’ (Gartner) accessed 23 April 2024.

(11) Tim Keary, ‘AI Hallucination’ (Techopedia.com) accessed 23 April 2024.

(12) ‘Glossary’ (University of Oxford; Saïd Business School; GetSmarter); Angie Lee, ‘What Are Large Language Models Used for and Why Are They Important?’ (NVIDIA Blog 26 January 2023) accessed 24 May 2023.

(13) Margaret Rouse, ‘What Is Training Data? – Definition from Techopedia’ (Techopedia.com) accessed 23 April 2024.