FEM-Bench: A Structured Scientific Reasoning Benchmark for Evaluating Code-Generating LLMs

This article introduces FEM-Bench, a new benchmark designed to assess the scientific reasoning capabilities of Large Language Models (LLMs) that generate code. The focus is on evaluating how well these models can handle structured scientific reasoning tasks. The source is ArXiv, indicating it's a research paper.