Abstract Summary/Description
The intricacies of accurate decision-making in cricket officiating, which require detailed rule interpretation and responsiveness to dynamic match events, pose a significant challenge for automation. This research introduces an innovative framework that equips a Pepper robot to serve as an intelligent cricket umpire, capable of automating scoring and decision-making in real-time. Through an ontology-driven, multi-modal methodology, our system leverages Graph-based Retrieval-Augmented Generation (Graph RAG) to produce rule-compliant, context-aware decisions that align with professional officiating standards. The framework is structured around two core functionalities: first, a text-based retrieval system that uses a structured ontology of cricket rules encoded in graph formats (e.g., JSON-LD, RDF) to provide precise rule retrieval, informed by pretrained language models fine-tuned on historical decision data. This functionality supports accurate, scenario-specific text-based decisions. The second functionality expands to video analysis, employing models such as I3D and TimeSformer to extract critical features like ball trajectory from visual inputs. Integrating Graph RAG, the system dynamically cross-references these visual insights with encoded rules, improving accuracy in complex cases, including LBW adjudications. A multi-modal transformer further unifies text commentary and video data, producing comprehensive interpretations that adhere to cricket regulations. Extensive evaluations reveal high levels of accuracy, precision, and F1-score, with ablation studies highlighting Graph RAG’s essential role in decision-making for ambiguous scenarios. This project highlights the potential for intelligent automation in sports officiating and will be tested in practice nets at parks across North Georgia.
