Automated Hypothesis Generation for Human Cooperation Studies

A meta-analysis is a systematic review of the literature that employs statistical techniques to combine and compare the results of multiple related studies, that can be used to test a hypothesis. In the social sciences, it is the gold standard for research synthesis. However, it requires considerable effort including hypothesis formulation and background research. In our work, we envision a human-AI collaboration process, where AI-generated research hypotheses are integrated in tools that empower domain experts while keeping them in control, fostering trust and efficiency in social science research. In this work, we propose using knowledge-enriched AI methods as an alternative to hypothesis generation to be tested by the meta-analysis, on the specific case study of human cooperation. We collaborated with domain experts to formulate three template hypotheses of increasing complexity. We then compared three knowledge-enriched AI methods, classification, link prediction, and large language model, for hypothesis generation. To evaluate the quality of the hypotheses generated, we conducted user studies with domain experts, comparing the hypotheses they generated with those generated by AI methods. We demonstrate that AI-generated hypotheses, though not always surpassing human-generated ones, provide diverse, efficiently produced options that effectively complement and enhance the hypothesis generation process.