AI tools such as ChatGPT will disrupt hydrology, too

Recent advances in AI have led to a variety of powerful tools that are rapidly upsetting business-as-usual across a range of industries. ChatGPT (OpenAI, 2022) is a pre-trained natural language processing (NLP) model that interacts with users via text. In the short time since its release, ChatGPT has provoked a frenzy of hyperbolic reactions, ranging from excitement to trepidation, in the scientific and popular media. It represents the current state-of-the-art in NLP and is capable of producing convincing and well-articulated responses to a wide range of prompts. Because of its strong capacity for human-like conversation output and its massive and diverse training dataset, the potential implications of ChatGPT, as well as forthcoming AI tools, merit thorough examination by all hydrologists involved in research or education.

AI tools such as ChatGPT can assist hydrologists in many aspects of their work. While there is justified debate about the ethics of using AI tools in scientific authorship, several tasks that fall short of actual authorship are likely to see widespread use and, we posit, eventual acceptance. Hydrologists must have an open mind when they consider the ability of AI tools to translate, consolidate and improve text (and code), taking solace in the knowledge that original, novel research will still require the human mind for the foreseeable future. While many hydrology investigations are, at first glance, ‘local’, the impact and reach of the most high-quality research are global. Considering that the degree of fluency in English, the de facto language of science, varies greatly among both authors and readers, the potential positive impacts of AI tools in hydrological research and publishing cannot be overstated.

ChatGPT's ability to ‘pass’ various exams has recently been investigated. ChatGPT's performance on the US Medical Licensing Exam questions was recently deemed equivalent to that of a typical third-year medical student (Gilson et al., 2022). The tool was also capable of obtaining a ‘B-level’ grade on an MBA exam in Operations Management (Terwiesch, 2023). Notably, ChatGPT was remarkably strong at providing eloquent answers to case-study process-based questions and was able to refine its answers when given short ‘hint’ prompts.

In many areas of hydrological geosciences, machine learning (ML), a subset of AI, is commonplace (Lange & Sippel, 2020; Xu & Liang, 2021). ML techniques include algorithms such as the k-nearest-neighbours (a classifier) and principal component analysis (a dimensionality reducer) that are well-known to hydrologists, especially geostatisticians and those working with geochemical or remote sensing data. However, despite decades-long interest in topics such as artificial neural networks for hydrological forecasting, advanced AI techniques have not yet seen widespread uptake by a broad spectrum of the hydrology community and are often viewed with scepticism by process-focused hydrologists. With the exception of search engines, ChatGPT is likely to be the first AI tool that sees mainstream use in our domain and will have correspondingly far-reaching impacts.

1 CAN CHATGPT SOLVE BASIC HYDROLOGICAL PROBLEMS?

While the ability of ChatGPT to answer assignment and exam questions has been reported on for some domains (Gilson et al., 2022; Kortemeyer, 2023; Terwiesch, 2023), no investigation of its capabilities in hydrological sciences has been reported. To test such capacities, we imput a variety of quantitative and qualitative questions based on two hydrogeology textbooks (Fetter & Kreamer, 2022; Fitts, 2013). We also interacted informally with the tool to further explore and evaluate its capacities. The quality of ChatGPT's answers to hydrological questions varies greatly. There are marked differences between its ability to answer qualitative and quantitative questions and we note that ChatGPT often makes mistakes in unexpected parts of its answers.

Among the most impressive responses from the AI tool are its answers to qualitative questions, although there are notable caveats. ChatGPT is remarkable in its ability to eloquently summarize and express information. When posed an ‘open’ style question such as listing the positive impacts of groundwater well development (Q11, SI¹), the tool provides a thorough list with detailed explanations for each entry that would surely receive a top mark from a human evaluator. On the other hand, the tool can state information that is blatantly false and can contradict itself in a single response. For example, enquiries about stable isotopes resulted in explanations that were mostly correct but included descriptions of ¹⁸O as the light isotope and ¹⁶O as the heavy one. We also noticed problems in reasoning (e.g., Q13, SI) that gave the impression the tool were confusing multiple concepts. ChatGPT is far from perfect in terms of hydrological processes knowledge, although the overall quality of its responses to questions that do not involve calculations is good enough that students, educators and researchers should take keen notice.

When it comes to answering quantitative questions, ChatGPT is less convincing. Answering a typical quantitative question in an undergraduate hydrology course might involve three general steps: (1) understanding and assembling the available information, (2) identifying the appropriate equations, and (3) numerically evaluating the answer. ChatGPT is very strong in the first step and, moreover, explicitly identifies any information that is irrelevant. Nonetheless, it is not infallible in this aspect and can sometimes misinterpret the meaning of parameters, units, dimensions, and other quantities. The second step is where the tool often makes a mistake by choosing an inapplicable mathematical relation, albeit one that has some similarities to the correct one. For example, if provided with a question about groundwater abstraction in an unconfined aquifer (explicitly defined as such), ChatGPT proposes the Theis equation, which is only valid for confined aquifers. This type of issue resurfaces often and ChatGPT's ability to decide on a suitable analytical solution for a given problem appears to be a major point of weakness. The final step in solving a typical quantitative question involves determining the answer, which may be numerical if sufficient information is given. ChatGPT appears to reliably perform simple rearrangements of equations, correctly solving for relevant variables, yet it often fails in the numerical evaluation of formulae. Indeed, when asked a very simple question about the volume of a cylindrical reservoir (Q1, SI), ChatGPT failed to evaluate the answer, despite appearing to understand the meaning of the variables and knowing the correct equation. It also regularly fails in unit conversion. Seeing as handheld calculators have been in use for decades for these trivial steps, we find this point of weakness rather surprising.

Any hydrology student hoping to use ChatGPT to complete a hydrology assignment or open-book exam will surely be disappointed—at least for the time being. Fortunately, there is still a significant need for the human brain, although if generous partial marks are given, it is plausible that ChatGPT could receive a passing grade on an undergraduate-level hydrology assignment. We strongly recommend that hydrology educators explore the tool's capabilities and limitations for themselves to understand its capabilities in their specific domain.

2 HOW CAN HYDROLOGY RESEARCHERS AND EDUCATORS USE CHATGPT?

While much of the focus by educators on ChatGPT and other AI tools concerns their ability to complete assignments and exams (i.e., the threatening aspects of the technology), there are numerous ways in which they can be used as tools for education and research. Indeed, this rapidly evolving technology has the potential to make hydrologists more efficient in various aspects of their work without compromising pedagogical or scientific quality. In understanding its limitations and capabilities, hydrologists and other scientists have much to gain from ChatGPT and other similar tools.

Educators can take the above-discussed threat of students using ChatGPT to ‘cheat’ and turn it on its head by using the tool to assist with the creation of exam or assignment questions. While it cannot know exactly what is covered in a particular course, given adequate prompts and information, ChatGPT can produce various types of questions suitable for exams or assignments. For example, the tool can be prompted to produce variations on existing questions or to provide multiple-choice answers to a given prompt. For graded assignments, one can simply ask ChatGPT to answer its own question—if its answer is incomplete or wrong, then it is reasonable to assume that students will not be able to ‘cheat’ by using ChatGPT. Nonetheless, as the tool is constantly learning, it is possible that a correct or, at least, more complete response may be generated by the tool at a later time.

In our opinion, the most powerful use of ChatGPT for hydrology researchers is its ability to consolidate, expand, and improve text. This is, correspondingly, its most controversial aspect and has already led to blanket bans by publishers (Thorp, 2023). For example, authors can write a point-by-point summary of what they intend to discuss in a section and subsequently have ChatGPT convert the text into paragraphs of specified lengths. Such capabilities are the source of a growing debate about the role of AI in scientific writing (Stokel-Walker, 2023). In spite of the fact that ChatGPT may have been trained on a colossal amount of data—ostensibly including a large amount of hydrological research—it certainly does not yet appear close to being able to write a convincing review article on a hydrology topic, let alone carry out original process-based hydrology research. Furthermore, those who take an overly apprehensive view of AI tools in scientific writing miss the fact that these tools can help drastically improve writing that is scientifically sound but lacks readability. As the majority of hydrology research is written in English by non-native speakers, this has many positives for authors, readers, and editors alike. Finally, a similar, but less controversial, capability of ChatGPT and other emerging AI tools is the writing, debugging, and translation of code (Finnie-Ansley et al., 2022). This aspect should be fully embraced by the hydrology community with the caveat that the results, much like any AI-assisted text, need to be critically evaluated.

3 WELCOME OR NOT, AI WILL DISRUPT HYDROLOGY

ChatGPT is likely to cause headaches for editors and reviewers in the coming months and years. This is not because it is as good as humans at critically analysing and presenting science (it is not), but rather because it is remarkably adept at producing ‘bullshit’ (Frankfurt, 2005)—that is, statements that are not necessarily untrue but are superficially convincing and often devoid of meaning. This type of writing can be excruciating for readers and often requires multiple readings to identify. To appreciate this aspect, we encourage readers to write a technical sentence about a hydrology topic and to ask ChatGPT to expand that statement to 500 words. They will likely observe significant repetition, near-empty statements, and tangentially-related information, all stated with perfect grammar and syntax. One can imagine that part of the reason ChatGPT is so good at producing this type of writing is that its training data included significant amounts of it.

ChatGPT is not static and, with continuous development and ever-growing quantities of training data, its responses to given prompts will evolve and, ostensibly, improve. This renders blanket bans on its use in higher education unenforceable and, therefore, inadvisable. Hydrology and geoscience educators in general need to be aware of its capabilities and weaknesses when designing assignments and an emphasis should be made on promoting critical thinking rather than memorisation. This advice is not new (Bradforth et al., 2015) and holds regardless of the level of sophistication of AI technology. ChatGPT does reasonably well at answering hydrology questions, but falls short in many key aspects. Thus, at this point, educators in our domain should not worry excessively about it, but should nevertheless take time to acquaint themselves with the tool and, potentially, employ it in their teaching. Knowing this, we believe that ChatGPT and other AI tools to come are likely to offer more opportunities than threats for the hydrology community.

For hydrology researchers, it is clear that ChatGPT and other AI tools have the potential to be truly disruptive. The debate about AI's place in scientific authorship will continue to intensify—justifiably so. However, a hydrologist's critical thinking skills are still very far from being made obsolete. One must be clear that ChatGPT cannot collect field samples, build hydrological models, solve site-specific problems, or even adequately cite current literature. In short, AI cannot (yet) truly ‘do’ hydrology and, as such, should be viewed primarily as an exciting new tool, rather than a threat, by our community.